Synthetic data-driven hemodynamic determination in medical imaging

ABSTRACT

In hemodynamic determination in medical imaging, the classifier is trained from synthetic data rather than relying on training data from other patients. A computer model (in silico) may be perturbed in many different ways to generate many different examples. The flow is calculated for each resulting example. A bench model (in vitro) may similarly be altered in many different ways. The flow is measured for each resulting example. The machine-learnt classifier uses features from medical scan data for a particular patient to estimate the blood flow based on mapping of features to flow learned from the synthetic data. Perturbations or alterations may account for therapy so that the machine-trained classifier may estimate the results of therapeutically altering a patient-specific input feature. Uncertainty may be handled by training the classifier to predict a distribution of possibilities given uncertain input distribution. Combinations of one or more of uncertainty, use of synthetic training data, and therapy prediction may be provided.

RELATED APPLICATIONS

The present patent document claims the benefit of the filing date under35 U.S.C. §119(e) of Provisional U.S. Patent Application Ser. No.62/083,373, filed Nov. 24, 2014, which is hereby incorporated byreference.

BACKGROUND

The present embodiments relate to computation of blood flow in a vesselof a patient. In particular, a hemodynamic metric is estimated fromnon-invasive medical imaging data.

To estimate a value for flow, a computer model of the vessel is used.For flow in a particular patient, an anatomical model is fit to imagingdata for that patient. Computational fluid dynamics estimates the flowfrom this patient-specific model. However, this approach relies only ongeometrical information available from the medical imaging data.

In other approaches, machine learning is used. Either medical images orgeometric models extracted from imaging data populate the trainingdatabase. Features are extracted from these examples for training. Theground truth blood flow measurements are from the patient orcomputational fluid dynamics measurements. Machine training is performedto create a classifier able to estimate the blood flow from the inputfeatures. Due to reliance of patient-specific information, the machinelearning may be limited. The training data should include as manyexamples as possible, such as hundreds or thousands of examples. Giventhe broad variability in the patient population, an even greater numberof examples should be gathered for training. The availability of suchexamples is limited. The cost and time to gather sufficient trainingdata is a detriment and outlier conditions are less likely to beaccounted for in the machine-learnt classifier.

BRIEF SUMMARY

By way of introduction, the preferred embodiments described belowinclude methods, computer readable media and systems for hemodynamicdetermination in medical imaging. Rather than relying on training datafrom other patients, the classifier is trained from synthetic data. Acomputer model (in silico) may be perturbed in many different ways togenerate many different examples. The flow is calculated for eachresulting example. A bench model (in vitro) may similarly be altered inmany different ways. The flow is measured for each resulting example.The machine-learnt classifier uses features from medical scan data for aparticular patient to estimate the blood flow based on mapping offeatures to flow learned from the synthetic data. Perturbations oralterations may account for therapy so that the machine-trainedclassifier may estimate the results of therapeutically altering apatient-specific input feature. Uncertainty may be handled by trainingthe classifier to predict a distribution of possibilities givenuncertain input distribution. Combinations of one or more ofuncertainty, use of synthetic training data, and therapy prediction maybe provided.

In a first aspect, a method is provided for hemodynamic determination inmedical imaging. Medical scan data representing a vessel structure of apatient is acquired. A set of features are extracted from the medicalscan data. A first of the features of the set are modified to representa change to the vessel structure due to therapy. An uncertainty isassigned to the first feature of the set. A processor inputs thefeatures to a machine-trained classifier. The features include the firstfeature after the modifying and with the uncertainty. The machinetrained classifier is trained only from synthetic data not specific toany patients. With application of the machine-trained classifier, theprocessor outputs a hemodynamic metric with a confidence interval fordifferent values of the hemodynamic metric.

In a second aspect, a method is provided for hemodynamic determinationin medical imaging. A plurality of examples of vessel arrangements aregenerated with computer modeling, physical modeling, or both computerand physical modeling. A value for a flow characteristic is stored foreach of the examples of the vessel arrangements. An input feature vectoris extracted from each of the examples. A classifier is trained withmachine learning from the input feature vectors and the values for theflow characteristic of the vessel arrangements.

In a third aspect, a system is provided for hemodynamic determination inmedical imaging. A scanner is configured to scan a vessel of a patient.A memory is configured to store a plurality of features of the vessel ofthe patient, the features determined from the scan of the vessel. Aprocessor is configured to modify a first feature of the features froman abnormal state to a therapeutically corrected state, to apply thefeatures including the first feature as modified to a machine-trainedpredictor trained with training data of examples of vessels in thetherapeutically corrected state, and to output a prediction of a valueof a hemodynamic variable based on the application of the features tothe machine-trained predictor. A display is configured to indicate thevalue of the hemodynamic variable in association with thetherapeutically corrected state.

The present invention is defined by the following claims, and nothing inthis section should be taken as a limitation on those claims. Furtheraspects and advantages of the invention are discussed below inconjunction with the preferred embodiments and may be later claimedindependently or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

The components and the figures are not necessarily to scale, emphasisinstead being placed upon illustrating the principles of the invention.Moreover, in the figures, like reference numerals designatecorresponding parts throughout the different views.

FIG. 1 is a flow chart diagram of one embodiment of a method forhemodynamic determination in medical imaging;

FIG. 2 is a flow chart diagram of another embodiment of a method forhemodynamic determination in medical imaging;

FIG. 3 is an example virtual angiogram;

FIG. 4 is an example intensity as a function of time curve;

FIG. 5 illustrates a synthetic vessel model;

FIG. 6 illustrates bifurcation asymmetry and bifurcation angle;

FIG. 7 illustrates a synthetic model of a stenosis;

FIG. 8 shows an example normal distribution of radius of an arterialsegment;

FIG. 9 illustrates an example vessel tree;

FIG. 10 illustrates an example progression of synthetic creation of abifurcation stenosis;

FIG. 11 shows example geometrical features describing a shape of astenosis;

FIG. 12 shows example region of interest identification on a vessel;

FIG. 13 shows example placement of distal and proximal regions ofinterest on vessels visualized in angiography;

FIG. 14 shows example graphs of Savitzky-Golay filtering for proximaland distal regions of interest;

FIG. 15 illustrates an example of gamma variate filtering of a timedensity curve;

FIG. 16 is a flow chart of one embodiment of a method for calculatingischemic weight for a coronary artery segment;

FIG. 17 shows an example coronary tree labeled by segment;

FIG. 18 shows an example coronary tree with ischemic weights andlongitudinally varying cross-sectional radii in a healthy anatomicalmodel;

FIG. 19 shows an example partially diseased vessel segment andcorresponding ischemic contribution score;

FIG. 20 shows an example computation of ischemic contribution score fora bifurcation lesion;

FIG. 21 illustrates an example use of predicated hemodynamic metric atupstream locations as a feature to predict the hemodynamic metric at adownstream location;

FIG. 22 illustrates an example of flow interaction between vessels;

FIG. 23 shows an example vessel tree with stenosis on side branches;

FIG. 24 is an example angiographic projection for two-dimensional vesselsegmentation;

FIG. 25 illustrates example annotation for distance;

FIG. 26 illustrates an example annotation of a centerline;

FIG. 27 illustrates example template options selectable for differentsynthetic representation of the anatomy;

FIG. 28 illustrates one example of an in vitro model for generatingsynthetic data;

FIG. 29 shows an example segment with a high ischemic contributionscore;

FIGS. 30A and 30B show one embodiment of regular or continuouscomputation of a hemodynamic metric while processing;

FIG. 31 is an example display of a hemodynamic value at a user selectedlocation;

FIG. 32 is an output according to one embodiment with color coding;

FIG. 33 is an example fly-through visualization;

FIG. 34 is an example unfolded view of the vessels in an arterial tree;

FIG. 35 is an example output where particles are represented withstatistical information;

FIG. 36 is an example vessel visualization for different hemodynamicmetrics on a path inside the vessel;

FIG. 37 is an example vessel visualization with cross-sectioninformation;

FIG. 38 is a flow chart of one embodiment of a method for hemodynamicdetermination in medical imaging using uncertainty;

FIG. 39 illustrates one embodiment of a method for updating syntheticdata and a machine-learnt classifier;

FIG. 40 is a flow chart of one embodiment of a method for hemodynamicdetermination in medical imaging using sequential learning;

FIG. 41 illustrates modification due to virtual therapy;

FIG. 42 illustrates automatic detection of proximal and distal planes ofa stenosis;

FIG. 43 is a flow chart of one embodiment of a method for hemodynamicdetermination in medical imaging using therapy modification;

FIG. 44 is a flow chart of one embodiment of a method for hemodynamicdetermination in medical imaging from one physiological state toanother;

FIG. 45 is a flow chart of one embodiment of a method for hemodynamicdetermination in medical imaging using reduction in order of themodeling;

FIG. 46 shows an example comparison of machine-learnt as opposed tocomputational fluid dynamics computation of a hemodynamic metric; and

FIG. 47 is a block diagram of one embodiment of a system for hemodynamicdetermination in medical imaging.

DETAILED DESCRIPTION OF THE DRAWINGS AND PRESENTLY PREFERRED EMBODIMENTS

A data-driven approach provides for hemodynamics computation. Theapproach includes a machine-training phase and a prediction phaserepresented in FIG. 1. The training phase is an offline process, duringwhich a database of synthetically generated geometries withcorresponding hemodynamic metrics is first assembled in acts 12 and 16.In this database, a number of features that characterize the geometry orother characteristics represent each sample. These features areextracted in act 12. The mapping between the features and thehemodynamic metric is learnt in act 14 using a machine-learning basedalgorithm.

The prediction phase is an online process. The data for a specificpatient is loaded in act 18. The required features are extracted fromthe new patient dataset in act 20. The values of the features are thenused as an input to the pre-learned model. The machine-learnt classifiercomputes the value of the hemodynamic metric for new patient data (e.g.,unseen data) in act 22. The learned mapping from the training phase isapplied to the patient data. The machine-learnt computation ofpatient-specific hemodynamic metrics uses patient-specific geometricalfeatures despite being trained on synthetic data.

FIG. 2 shows another workflow or method for computing patient-specificcoronary measures. Patient-specific medical imaging information is usedto determine a hemodynamic metric or metrics. To predict one or morehemodynamic indices, a surrogate model is trained in act 14 using amachine learning approach. For training, a database 28 of just syntheticarterial trees is generated as training data in act 10. The database 28is a general database. Alternatively, the database 28 is specific to aninstitution, such as having been created under the control of theinstitution. The synthetic examples are generated in silico or in vitro.In act 16, computational fluid dynamics (CFD) computations for the insilico anatomical models or flow experiments for the in vitro anatomicalmodels are performed to determine a ground truth or value of thehemodynamic metric for each example. Depending on the metric or metricsof interest, one or more measures of interest are extracted in act 24.For instance, for coronary hemodynamics these indices may be fractionalflow reserve (FFR), coronary flow reserve (CFR), instantaneous wave freeratio (iFR), and/or related quantities.

In parallel, geometric and/or other features are extracted from thesynthetic examples, such as from anatomical models, in act 12. In act14, a data-driven surrogate model(s) is trained using the geometricfeatures and the target measure(s).

Once the surrogate model has been trained, the measures of interest maybe predicted in act 22 for patient-specific geometries obtained frommedical images (X-ray angiography, computed tomography angiography,magnetic resonance, or other scan) and/or other data. Thepatient-specific vessel geometry is extracted in act 26. Either the samefeatures as for the synthetic data or a subset of the features areextracted in act 20 from the vessel geometry. These features are used asinput data for the surrogate model.

If the patient-specific data does not include one or more features, themissing features may be either predicted from a separate machine-learntmodel in act 32 or estimated using similar anatomies in the database 28of synthetic geometries in act 30.

For training and for prediction (i.e., application of the machine-learntclassifier), features are extracted. The same set of features areextracted from the medical scan data and/or other patient specific datafor application of the classifier as are used for training theclassifier. The machine training may determine more discriminativefeatures, so may provide a classifier that uses fewer of the featuresfor prediction. For the discussion below on feature extraction, the sameor different process is used to extract features from the synthetic datafor training and for the patient-specific data for prediction.

In acts 10 and 26, coronary arterial trees or other vessel structuresare extracted from data. For the generating from synthetic data, theextraction may be in the form of altering an existing model, creating amodel that is not directly extracted from a medical scan. For generatingfrom patient-specific data, the extraction is from medical scan datarepresenting the vessel in two or three dimensions.

In act 10, to train a surrogate model using the machine learningapproach, only synthetically generated geometries (vessel trees) areused. The synthetic geometry used during the training phase is either afull vessel tree or some part of the full vessel tree. In otherembodiments, the geometry is of a single segment or branch of the vesseltree.

A starting model may be created from a given patient, but the majorityof training examples are based on alterations from the starting model.Alternatively, the starting model or models are averages or other modelsnot directly related to a given patient. The data is synthetic by notbeing extracted from data for particular patients. The synthetic vesseltree may have either a physical (in vitro) or a digital (in silico)representation. The digital representation is generated and stored on acomputer. In alternative embodiments, some or a majority of the trainingexamples are extracted from patient-specific data for a plurality ofpatients and only some of the examples are alterations of those models.If real patient anatomies are available, further synthetic trees may beconstructed by stochastically perturbing the features of the patientanatomy. This added synthetic data may be used to get a richerrepresentation, which can account for uncertainties in the data.

The in vitro synthetic models are three dimensional vessel treesartificially modeled with tubes or other devices. The in silico modelsare either full-scale (three dimensional) or reduced-scale models (two,one or zero-dimensional models). The number and nature of parameters andthe configuration of a synthetic in silico geometry may depend on themodel fidelity or scale. The highest level of detail is used for fullscale models. The geometry is represented by a three-dimensional mesh, amask, a cloud of three-dimensional points representing the arterialwalls, or any other representation which describes the lumen of thevessel tree. A centerline tree may be used as input data for generatingthe three-dimensional mesh or the point cloud. For a two-dimensionalrepresentation, the lumen boundaries are represented by lines instead ofsurfaces or by a cloud of points. In a one-dimensional representation,the centerline and the effective radius at each centerline point areprovided. The centerline may be represented in a one-, two-, orthree-dimensional space. For a zero-dimensional in silico model, thevessel tree is represented by one or several lumped segments, whereaseach segment is described by a series of parameters (e.g. resistance,compliance, or inertance) along with further parameters describing theinteractions between different segments. The reduced-scale models may bedetermined from full-scale models by extracting the relevant informationor may be generated directly.

To populate the database 28 in act 10, different approaches may be used.One or more baseline models, whose properties are then randomly orsystematically perturbed to obtain a large number of models, arecreated. The baseline models may be represented by healthy populationaverage coronary geometries, atlas models, and/or animal data. Otherbaseline models may be used.

In another approach, each model is generated separately by following aset of rules and by randomly or systematically perturbing the parametervalues of these rules. Scaling laws may be used for generating realisticsynthetic models.

The generation of synthetic data may include generating syntheticimages, such as represented in FIG. 3. The synthetic image isartificially created to be similar to those obtained from differentimaging modalities (angiography, computed tomography (CT), ultrasound(e.g., Echo), or other). The synthetic geometries are then extracted inact 10 from these synthetic images using the same techniques as in thecase of real patient images.

FIG. 3 shows an example virtual angiogram generated to mimic aninterventional exam. The virtual angiogram may then be further used toextract features related to contrast agent propagation for the syntheticgeometries. For example, time density curves, transit time, bloodvelocity, blood flow rate, and/or other features may be determined inact 12 directly from the artificial image or from vessel geometryextracted from the artificial image.

FIG. 4 shows a time density curve extracted from a virtual angiogram.The time density curve includes various features: t_(fa) (time of firstappearance), t_(hm) (time to half of the peak opacification), t_(pk)(time of peak gradient), and t_(po) (time to peak opacification).Additional, different, or fewer features may be used.

In another embodiment, synthetic feature vectors are extracted directlyin act 12 instead of first generating synthetic geometries, from whichthe feature vectors are then extracted. An algorithm generates thehemodynamic metrics of interest in act 24 without act 16 as well asgenerates the feature vector in act 12. This algorithm may use thedatabase 28 in which synthetic geometries are mapped with syntheticfeature vectors and learn how to generate directly synthetic vectors.

The large number of variations available is one benefit of usingsynthetic data for training. Additional examples for training arecreated by altering one or more values of variables for the geometricstructure and/or for generating the geometric structure. Any number ofdifferent parameters may be varied. Hundreds or thousands of differentexamples for training may be generated from a single starting model.

FIG. 5 shows an example vessel tree model and corresponding parametersthat may be varied. Any number of degrees of freedom, step size invariance, or variance patterns for a given variable or combinations ofvariables may be used.

One parameter is the radius. The radius may be varied independently ateach location or systematically along a vessel segment. Limitations onthe variance may be provided, such as imposing a certain degree ofvessel tapering. A reference radius value may be defined for each vesselsegment. The reference radius is used as baseline for determining theradius at each location along the segment. Different rates of taperingmay be used. Other geometric features which characterize local vesselsize, such as the area, or effective (hydraulic) radius can also beused. The length of each vessel segment where a vessel segment isdelimited by bifurcations may be varied. The vessel curvature may bevaried.

Bifurcation parameters may be varied. The relationship between the radiivalues of the vessel segments connected at a bifurcation is varied. Forexample, a power law may be used at the bifurcations to describe theradiuses of the two daughter vessels:r _(p) ^(ξ) =r _(d1) ^(ξ) +r _(d2) ^(ξ),where the subscripts p, d1 and d2 refer to the parent vessel, and thetwo daughter vessels, respectively. The bifurcation may have more thantwo daughter vessels, in which case the model may be adapted asrequired. Different values for laminar flow, ξ, varying between 2.0 and3.0 may be used.

Generally, the bifurcations are considered to be asymmetric, and hencethe radii of the daughter vessels may be determined based on the radiusof the parent vessel by using two parameters represented in FIG. 6:r _(d1) =αr _(p) , r _(d2) =βr _(p),where α and β are two scaling parameters for bifurcation asymmetry. Twoadditional parameters are introduced, namely the area ratio and theasymmetry ratio respectively, defined as:

${\eta = \frac{r_{d\; 1}^{2} + r_{d\; 2}^{2}}{r_{p}^{2}}},{\gamma = {\left( \frac{r_{d\; 2}}{r_{d\; 1}} \right)^{2}.}}$The parameters ξ, η and γ are interconnected through the relationship:

$\eta = {\frac{1 + \gamma}{\left( {1 + \gamma^{\xi/2}} \right)^{2/\xi}}.}$Thus the two scaling parameters can be computed as:α=(1+γ^(ξ/2))^(−1/ξ), β=α√{square root over (λ)}.Other parameters and/or parameters of the geometry may be varied.

FIG. 6 shows another bifurcation parameter that may be varied to createadditional examples for the database 28. The bifurcation angle isvaried.

The presence, number and location, or the absence of side branches thatdraw the blood away from the main branches is varied. The side branchesmay have a major impact on the hemodynamic metric of interest, since theblood flow distribution in the entire geometry is modified. Vessel wallproperties may be varied. For instance, the wall may be modeled asrigid, elastic, viscoelastic or other formulations. Depending on themodel used for representing the vessel wall, different properties mightbe set, like wall thickness or Young's modulus. The presence and/orabsence of pathologic segments (e.g. stenoses, aneurysms, coarctations,or nature of plaque) may be varied.

The location of pathologic segments may be varied. The properties ofpathologic segments may be varied. These properties depend on thespecific pathology considered for each geometry. For example, ifatherosclerosis is considered for a synthetic model, several stenosesmay be placed along the vessel tree. Different types of stenoses may begenerated: single segment/bifurcation stenoses, focal, long, diffuse, orother.

To generate these various types of stenoses, various stenosis propertiesmay be used. FIG. 7 shows one example set of stenosis parameters to bevaried, but other parameter sets may be used. The percentage reductionof the radius at the location with minimum radius, total length, entrylength, exit length, percentage diameter stenosis, tapering betweenstart and end of stenosis, relative length of the region with minimumstenosis radius compared to the stenosis length, relative position ofthe location with minimum radius compared to the location of the centerof the entire stenosis, inlet angle, outlet angle, eccentricity,curvature, presence and extent of calcification or plaque, and/ormorphology of the plaque—for instance, lipid, fibrous, calcified ornecrotic, may be varied.

The parameters described above and/or other parameters may be modifiedto produce more pathological cases in the database 28. The values of theparameters are either chosen randomly for each synthetic example (e.g.,true or false for binary variables or a value in a predefined range forcontinuous variables) or the entire parameter space is exploredsystematically within limited ranges when generating the database ofsynthetic examples. Any type of distribution may be used for thecontinuous variables, such as uniform, normal, or other. FIG. 8 shows anexample of the normal distribution of the root radius of a coronary leftarterial tree. Known, estimated, or standard normal distributions may beused. The synthetic examples generated are assigned the value for theroot radius of the coronary left arterial tree based on the distribution(e.g., probability of a given value per example assigned using thedistribution).

Other sources of variability may be used to create the syntheticexamples for training. Parameters characterizing the coronary morphologyare varied. Such parameters include type, characteristic, and/orpresence or not of calcification, plaque (e.g., fibrous tissue, lipidtissue, necrotic tissue, calcified tissue), thrombus existence, diffusedisease characteristic, total or sub-total occlusion, myocardialbridging (superficial and/or deep), congenital anomalies of coronaryarteries such as anomalous origin of a coronary artery from an abnormalsinus of Valsalva with an inter-arterial course between the greatarteries; anomalous origin of one coronary artery from the pulmonarytrunk, or others, aneurysmal dilatation and superimposedatherosclerosis, “high take off” coronary artery (i.e., the ostium isseveral millimeters above the sino-tubular junction (the artery may havea sharp downward angle and runs partially through the aortic wall)),myocardial bridging as either superficial or deep, coronary fistula,coronary artery dissection, coronary vasculitis as rheumatoid arthritis,systemic lupus erythematosus (SLE) or Behçet's disease, Kawasakidisease, polyarteritis nodosa, or persisting (post) inflammatoryaneurysms, fibromuscular dysplasia, coronary microembolisation, and/orleft or right dominance. Additional, different, or fewer morphologyparameters may be used.

FIG. 9 shows one training example of a synthetic vessel generated froman atlas. The atlas model is represented as a list of vessel segments,whereas each segment is linked to its parent and daughter segments, andthe type of each segment is set to either main or side branch segment.The synthetic geometries may be generated algorithmically from the atlasin different ways. As one example, an algorithm recursively generates aone dimensional representation of the synthetic model. First, if thecurrent segment is the root segment of the synthetic model, the startradius of the segment is computed. Next, if the segment is a side branchsegment, a random binary variable is used to determine if the currentsegment should be used in the current synthetic model or not. Next, thelength and the tapering level of the vessel segment are set using achosen distribution function, and based on these values the bottomradius of the segment is computed. The centerline and the radius at eachlocation along the centerline are then determined. Afterwards, anotherrandom binary variable is used to determine if a stenosis should begenerated or not for this vessel segment. If a stenosis is placed on thecurrent segment, the properties of the stenosis are set randomly fromthe available parameters. Finally, if the current segment has daughtersegments, the function is called for each daughter segment so as totraverse the entire atlas model. A sample algorithm for this approach ofgenerating synthetic geometries is presented below

generateRandomSyntheticGeometry(currentSegment)  if(currentSegment isroot segment)   currentSegment → topRadius = getRandomValue (r_(min),r_(max))  end_if  if(currentSegment is side branch segment)  if(currentSegment → getRandomBinary)    currentSegment →excludeFromGeometry()    return;   end_if  end_if  currentSegment →length = getRandomValue (length_(min), length_(max))  currentSegment →tapering = getRandomValue (taper_(min), taper_(max))  currentSegment →bottom Radius = computeBottomRadius (currentSegment → topRadius,currentSegment → tapering, currentSegment → length)  currentSegment →computeCenterlineAndRadiusAtEachLocation()  if(currentSegment →getRandomBinary() )   currentSegment → generateStenosis();  end_if if(currentSegment has daughter segments)   if(currentSegment →getRandomBinary() )    currentSegment → generateBifurcationStenosis();  end_if   currentSegment → computeRadiusOfDaughterSegments()  end_if for(each daughter segment of currentSegment)  generateRandomSyntheticGeometry(currentSegment → daughterSegment[i]) end_forOther programs using different or additional sequences, parameters, orprocess acts may be provided.

Once a synthetic geometry is generated, it may be further modified, suchas adapting the stenosis properties. FIG. 1 represents an exampleapproach for creating synthetic bifurcation stenosis, where the limitsfor the stenosis are decided either automatically or manually. A modelis used to deform the geometry. Once the bifurcation location isidentified as assigned a stenosis, the stenosis parameters are assigned,resulting in a given level and/or type of stenosis.

Other examples are generated by processing the same atlas again and/orby processing the resulting example as if an atlas. Other syntheticexamples may be created using other approaches, such as starting with athree-dimensional model. Rather than varying in steps, the parameters tobe varied may be randomly selected and then random values assigned.

Using synthetic modeling instead of requiring examples from a largecollection of patients for training data provides several advantages. Avery large number of cases may be automatically generated, leading to anextensive database. Complex pathological configurations may begenerated, such as serial stenoses, multi-branch stenoses, bifurcationstenoses, diffuse disease, or others, despite being rare among actualpatients. Rare pathological cases may be sampled better. Since thegeneration of synthetic in silico geometries may be completelyautomated, the cost of generating a large database is reduced ascompared to assembling patient examples. The examples may be extended todifferent demographic groups easily. The training may be done in aglobal manner or a site-specific manner, allowing the system to accountfor anatomical trends based on patient demographics and epidemiology.Finding sufficient examples in a local region may be difficult. Thetraining may be iteratively improved with either more data or withbetter representations of the features.

Once the synthetic geometries have been generated, the features whichare used for training the machine learning algorithm are extracted inact 12. The same features or some subset of the features are extractedfrom the medical images of the patient in act 20 and used for predictingthe hemodynamic metric using the trained model. Depending on the sourceand type of the input data, the extracted features may be binary,numerical, categorical, ordinal, binomial, interval, text-based, orcombinations thereof.

The extraction includes assigning features or calculating features. Forexample, a geometrical feature randomly generated for creating thesynthetic vessel tree is used as an extracted feature by assignment. Asanother example, a difference between two features is calculated fromthe created vessel tree.

Any type of features may be used. Morphological features may be used.The machine learning process may provide for certain features to be usedand others not to be used. To train, the features to be used may beselected by a programmer.

Some example features include the parameters used or selected to defineor create the vessel structure as described above. Other or differentfeatures may additionally or alternatively be extracted.

Geometric features of the vessel structure are extracted. Geometricfeatures characterizing the geometry of a stenosis may be extracted.Parameters characterizing the geometry of the stenosis include referencediameters (e.g., proximal and distal), minimal lumen diameter (MLD),lesion length (LL), minimum radius length (e.g., length of the stenosisin the region of minimum radius—a tolerance limit can be used fordetecting this region around the location with minimum radius), entranceangle, entrance length, exit angle, exit length, % diameter of stenosis(e.g., computed based on proximal and/or distal reference radii), or %area stenosis (e.g., computed based on proximal and/or distal referenceareas). FIG. 11 shows an example set of stenosis features. Additional,different, or fewer features may be extracted. Various combinationsobtained through algebraic, integration, or derivation operationsapplied for proximal, distal and minimum radius of the stenosis, or anyother stenosis-specific, may be additionally used.

Features may be extracted for the geometry of the branch bearing thelesion. Features characterizing the branch geometry include vesselradius sampled along the centerline, areas sampled along the centerline,terminal radius of the vessel tree, terminal area of the vessel treecenterline tortuosity measures, location of stenosis in coronary tree,cumulative or aggregated number of vessel narrowing proximal to thelesion, cumulative number of calcifications proximal to the lesion,and/or vessel type (e.g., left anterior decent (LAD), left circumflex(LCx), right coronary artery (RCA), diagond (D), optimum modulus (OM),and/or others). Additional, different, or fewer parameters may be used.

One or more coronary tortuosity measures may be used. Given a discretecurve as a set of points in three dimensions, a spline interpolation isfirst performed to determine a continuous curve C(x(t), y(t), z(t)),with t taking values between t₀ and t₁. Next, the following measures arecomputed:

Arc length:

${{{arcLength}(C)} = {\int_{t_{0}}^{t_{1}}{\sqrt{{x^{\prime}(t)}^{2} + {\gamma^{\prime}(t)}^{2} + {z^{\prime}(t)}^{2}}{\mathbb{d}t}}}};$Chord length:

${{{chordLength}(C)} = \sqrt{\left( {{x\left( t_{1} \right)} - {x\left( t_{2} \right)}} \right)^{2} + \left( {{y\left( t_{1} \right)} - {y\left( t_{2} \right)}} \right)^{2} + \left( {{z\left( t_{1} \right)} - {z\left( t_{2} \right)}} \right)^{2}}};$Curvature:

${{\kappa(t)} = \frac{{{r^{\prime}(t)} \times {r^{''}(t)}}}{{{r^{\prime}(t)}}^{3}}},$where r′(t)=(x′(t), y′(t), z′(t)) and r″(t)=(x″(t), y″(t), z″(t));Total curvature:

t_(c) = ∫₀^(arcLength)κ(s)𝕕swhere s is the arc length variable along the curve; andTotal squared curvature:

t_(s c) = ∫₀^(arcLength)κ²(s)𝕕sBased on these measures, many tortuosity measures may be defined, someof which are given as:

${\tau_{0} = \frac{chordLength}{{arc}\;{Length}}},{\tau_{1} = {\frac{{arc}\;{Length}}{{chord}\;{Length}} - 1}},{\tau_{2} = t_{c}},{\tau_{3} = t_{sc}},{\tau_{4} = \frac{t_{c}}{{arc}\;{Length}}},{\tau_{5} = \frac{t_{sc}}{{arc}\;{Length}}},{\tau_{6} = \frac{t_{c}}{{chord}\;{Length}}},{{\text{and/or}\mspace{14mu}\tau_{7}} = {\frac{t_{sc}}{{chord}\;{Length}}.}}$Additional, different, or fewer measures may be used.

Features characterizing the entire coronary tree may be extracted. Thefeatures for the coronary tree may include: left or right dominance,size of coronary territories and associated myocardial masses, terminalradius of each coronary branch, number of lesions, segments withlesions, bifurcations with any number of daughter vessels (e.g., typeand angulations), number and location of stents already implanted,and/or number and location of bypass grafts. Additional, different, orfewer features for the entire coronary tree may be used.

Other geometric features may be extracted. For geometric or otherfeatures, a set of naming conventions defining aspects of the vesselstructure are described. A centerline tree is constructed for a givencoronary arterial tree. The infinite number of points in the centerlinetree may be classified into a start point (i.e., the first point of thecenterline tree, corresponding to the ostium), zero, one or moreramification points (i.e.: a point where the centerline bifurcates intotwo or more centerline segments), an end point (i.e., a point for whichno further downstream centerline point exists), and interior points(i.e., points lying between a start/ramification point and aramification/end point). Each coronary segments are classified as a rootsegment (i.e., a segment delimited by a start and a ramification point),a branch segment (i.e., a segment delimited by two ramification points),or a leaf segment (i.e., a segment delimited by a ramification and anend point). Each coronary segment (e.g., root, branch, or leaf) islabeled as either a non-healthy segment (i.e., a segment that has anabnormal luminal narrowing or dilation) or a healthy segment (i.e., asegment that has no abnormal luminal narrowing or dilation). Othernaming conventions, classifications, or labeling may be used.

Other features extracted include parameters for one or moreabnormalities of the vessel structure. Abnormal morphology may becharacterized by characteristics of calcification, characteristics ofthe plaque (e.g., fibrous tissue, lipid tissue, necrotic tissue,calcified tissue), characteristics of thrombus, characteristics ofdiffuse disease, presence of total or sub-total occlusion, presence ofmyocardial bridging (superficial and/or deep), congenital anomalies ofcoronary arteries (e.g., anomalous origin of a coronary artery from anabnormal sinus of Valsalva with an inter-arterial course between thegreat arteries, anomalous origin of one coronary artery from thepulmonary trunk, or others), aneurysmal dilatation and superimposedatherosclerosis, “high take off” coronary artery (e.g., the ostium isseveral millimeters above the sino-tubular junction (the artery may havea sharp downward angle and runs partially through the aortic wall)),myocardial bridging: superficial and deep, coronary fistula, coronaryartery dissection, coronary vasculitis (e.g., rheumatoid arthritis,systemic lupus erythematosus (SLE), or Behçet's disease, Kawasakidisease, polyarteritis nodosa, and/or persisting (post) inflammatoryaneurysms), fibromuscular dysplasia, coronary micro embolization, and/orleft or right dominance. Additional, different, or fewer abnormalityfeatures may be used.

Functional features representing operation of the vessel structure maybe extracted. Functional information includes functional imaging, suchas measures of uptake, or other operational information, such ascontrast agent measures. For the training data, the functional featuresmay be determined from simulation, synthetically created images,modeling, and/or other representation of the operation of the vessel.

In addition to anatomic and morphological features from medical imagesor synthetic representation of a vessel tree, functional features mayalso be extracted. For example, data from a perfusion scan or othermedical imaging scan (e.g., single photon emission computed tomography(SPECT), positron emission tomography (PET), or perfusion imaging) mayalso be used to extract features such as metrics characterizing relativeand/or absolute tissue perfusion in each coronary territory at restand/or during stress. As another example, angiographic data maycharacterize contrast agent propagation. Some features characterize theflow of contrast at a given location, such as the time-to-peak tracerconcentration, and splits across different daughter vessels at abifurcation.

Some characteristics are extracted based on two regions of interest(ROI) defined for vessel trees. FIG. 12 shows two ROIs for a syntheticrepresentation of a vessel segment. Since direct measures of function(e.g., perfusion or transit time) are not available for in silicosynthetic data, modeling may be used. For in vitro synthetic data,direct measures, such as medical scan or measuring optically, may beused. Alternatively, one or more synthetic images are generated torepresent function. FIG. 13 shows distal and proximal ROIs on threevessels from synthetic or actual patient angiography scans.

One metric to be extracted is the transit time or the time required forthe contrast agent to traverse the distance between the two ROIs. Thetransit time may be estimated using manual, semi-automated, orfully-automated methods. Manual methods include counting the number offrames required for the contrast agent to traverse the distance betweenthe ROIs. Combined with the frame rate of the sequence, the transit timeis estimated. Semi-automated methods include manual placement of theROIs on each frame. Since the coronary vessels are continuously moving,the actual locations of the ROIs change from one frame to another. Thetransit time is automatically estimated from the manually placed ROIsand the data. The automated estimation of transit time is based on timedensity curves (TDCs). A TDC across a vessel's region of interest is thesurface integral of the pixels' intensities inside the ROI:D(t)=∫∫I(x,y,t)dxdywhere I(x,y,t) represents the pixels' intensities at the acquisitiontime t and D(t) is the time density curve.

Several noise sources may distort the shape of the time density curveand thus introduce errors in the estimation of transit time. The sourcesof noise include recirculation of the contrast material, extravascularaccumulation of contrast material that produces a lower peak and aslower washout, shape of the contrast bolus (especially for manualinjection), non-steady flows that may be observed when the contrastagent does not fully mix with the blood, and/or opacification ofbackground structures (bones). Before applying different methods fortransit time estimations, the computed time density curves arepost-processed through normalization, filtering and curve fitting. As anexample, a filtering with a Gaussian weighted moving average or aSavitzky-Golay filtering is used. FIG. 13 shows angiographic imagesafter Savitzky-Golay filtering. Other filtering may be used. FIG. 14shows Savitzky-Golay filtering for proximal ROI (left) and distal ROI(right) ROIs. Alternatively, a fitting of the time density curve may beperformed, identifying thus for example a complex exponential function(Gaussian or gamma variate function—Error! Reference source not found.)or a polynomial function that preserves the key characteristics of theslope (e.g., peak value, the delays of contrast appearance, and/or thewashout slope).

In one embodiment, the transit time is determined by selecting two ROIsalong a same vessel. The time density curves for the two ROIs areextracted. The time density curves are smoothed, such as withSavitzky-Golay or other filtering. A curve is fit to the time densitycurves. Any curve fitting may be used, such as fitting of a gammavariate function as represented in FIG. 15. The transit time isestimated based on the two time density curves (raw, smoothed and/orfitted). Various transit times may be used, such as:

mean transit time:

${t_{mtt} = \frac{\int_{0}^{\infty}{{t \cdot {D(t)}}{\mathbb{d}t}}}{\int_{0}^{\infty}{{D(t)}{\mathbb{d}t}}}};$mean transit time after curve thresholding:

${D(t)} = \left\{ {\begin{matrix}{{D(t)} - \Delta} & {{{if}\mspace{14mu}\mspace{14mu}{with}\mspace{14mu}\Delta} = {\alpha \cdot {\max_{t}{D(t)}}}} \\0 & {otherwise}\end{matrix};} \right.$time of peak opacification (i.e., the bolus is considered to havearrived at a ROI when the time-density curve reaches its peak value);time to half max (i.e., the bolus is considered to have arrived once thetime-density curve reaches half of its peak density);first appearance time (i.e., the bolus is considered to have arrivedwhen the density reaches 5% of its peak value) D(t_(fa))=0.05·D_(max);rise time: t_(rt)=t_(max)−t_(fa) where the reference time is t_(fa) (thefirst appearance time);mean concentration time (i.e., the bolus is considered to have arrivedwhen the density reaches the mean value for the first time);mean arrival time

$t_{mat} = {\frac{1}{D_{\max}}{\int_{t_{ref}}^{t_{\max}}{\left\lbrack {D_{\max} - {D(t)}} \right\rbrack{\mathbb{d}t}}}}$where t_(mat) is the mean arrival time, t_(ref) is the reference time,and t_(max) is the peak time;time of peak gradient (i.e., the bolus is assumed to have arrived whenthe gradient of the time-density curve reaches its maximum value);and/or cross correlation method (i.e., the time-density curve obtainedat the first ROI is shifted in time so that the curve superimposes thecurve obtained at the second ROI where the Δt value that maximizes thecross-correlation function φ(Δt)=∫₀ ^(t) ^(end)D_(ROI1)(t−Δt)·D_(ROI2)(t)dt is considered to be the time of bolustransport between the two ROIs. Additional, different, or fewer transittime features may be extracted.

Once the transit time is determined, other features may be estimated. Asexamples, the other features include: the velocity of the contrast agent(e.g., may be computed from the transit time and the distance betweenthe two ROIs along the centerline of the vessels), and/or the flow rateof the contrast agent may be computed from the transit time and thevessel volume between the two ROIs. Additional, fewer, or differentfeatures may be used.

Yet another example of features to be extracted are an ischemic weightand/or ischemic contribution. Some features based entirely on geometryinclude ischemic weight w and ischemic contribution score s. An ischemicweight value is associated to each coronary segment (root, interior orleaf segment). An ischemic contribution score is computed for a specificnonzero, finite length segment of coronary geometry, comprising one ormore branches. The ischemic contribution score is computed from a seriesof geometric properties and from the ischemic weights of the particularsegments.

For ischemic weight, the ischemic weight value, w, of each coronarysegment corresponds to the sum of the ischemic weight values of alldownstream segments. To compute the weights, a three steplocal-to-global-to-local approach shown in FIG. 16 is used. A separateischemic weight is computed for each branch in act 40. A local ischemiaweight value is estimated independently for each root/branch/leafsegment using geometric features of the segment, such as the referenceradius, length, tapering rate and other features. As an example, theischemic weight could be computed using:w=k ₁ ·r _(ref) ^(n),where, r_(ref) is the reference radius of the segment, k₁ is aproportionality constant, and n is a power coefficient. Since,regularly, the radius along the centerline of a segment, r(x), iscontinuously varying, a mathematical operator (f₁) is applied to computethe reference value:r _(ref) =f ₁(r(x)).An average value of healthy radiuses of the entire branch or a part ofthe branch, an average value of healthy radiuses obtained when excludingthe largest x % and the smallest y % of the radius values of the entirebranch or a part of the branch, or maximum or minimum value of healthyradii of the entire branch or part of the branch are computed.

As the local weights are computed independently, there is no guaranteethat the assumption that the sum of the ischemic weights of two daughterbranches is equal to the ischemic weight of the parent branch holds.Therefore, one global ischemic weight for the entire tree is computed byaveraging the weights of different branches in different generations.For example, a global ischemia weight value for the entire coronary tree(left or right coronary tree) based on ischemia weights w_(i) isdetermined. In act 42, a global ischemia value for each generation ofvessels is computed. FIG. 17 shows an example of a coronary tree where ageneration number, g, is attached to each branch. The root branch has ageneration number of 0, which then increases at each bifurcation by one.Before estimating the global ischemia weight, a confidence value c_(i)is attached to each branch. The confidence value represents theconfidence in the correctness of the computed reference radius or othergeometric parameter. Very short branches, such as the bottom branch withgeneration number equal to 1 in FIG. 17 or entirely diseased branches,such as the diffusely diseased branch with generation number equal to 2in

17 receive a low confidence value, while long vessels without radiusirregularities receive large confidence values. During modeldevelopment, other constraints may also be applied using knownoptimization methods.

In act 44, the global ischemia value for generation g is computed usinga mathematical operator f₂:(w _(global))_(g) =f ₂(c _(i) ,w _(i)),where index i refers to all branches of generation g and all terminalbranches with a generation number smaller than g. For example,(w_(global))_(g) is computed from:

$\left( w_{global} \right)_{g} = {\sum\limits_{i}{c_{i} \cdot w_{i}}}$Next, a final global weight value is computed from the individual globalweights (w_(global))_(g) corresponding to a single generation. Again, aconfidence value may or may not be attached to each generation, d_(j),and the final global weight value is determined using a mathematicaloperator f₃:w _(global) =f ₃(d _(j),(w _(global))_(j)),where index j refers to a generation number. For example, the globalweight is computed as a weighted mean:

$w_{global} = {\frac{\sum\limits_{j}{d_{j} \cdot \left( w_{global} \right)_{j}}}{\sum\limits_{j}d_{j}}.}$Other functions may be used.

In acts 46 and 48 of FIG. 16, the global ischemic weight is distributedto the individual branches in a way that satisfies the originalassumption. During the third step (local), starting from the globalischemia weight, a final local ischemia weight value is computed foreach root/branch/leaf segment. In act 46, the local weight of thecoronary leaf segments:

$w_{k} = {\frac{\left( r_{ref} \right)_{k}^{n}}{\sum\limits_{k}\left( r_{ref} \right)_{k}^{n}}w_{global}}$is computed, where k refers to the coronary leaf segments. Finally, theischemia weights of the branch and root segments are computed in act 48as a sum of the ischemia weights of all downstream leaf segments:

${w_{l} = {\sum\limits_{k}w_{k}}},$where k refers to all leaf segments lying downstream from the currentsegment l. Other functions may be used. Other representations ofischemic weight may be used.

Ischemic contribution score may be computed as a feature for a vesseltree. The ischemic contribution is a function of the ischemic weight anda geometric parameter, such as radius. The ischemic contribution scoreis computed for a nonzero finite length coronary artery segment that mayor may not contain ramifications.

The ischemic contribution score is computed differently for healthy andnon-healthy segments. Healthy segments have low ischemic contributionscores. For a healthy coronary artery segment, like the one in FIG. 18,the ischemic contribution score s is computed using the formula:

${s = {k_{2}{\int_{0}^{L}{\frac{w(x)}{{r(x)}^{n}}{\mathbb{d}x}}}}},$where L is the total length of the segment, k₂₁ is a proportionalityconstant, n is a power coefficient, r(x) is the radius that varies alongthe centerline, w(x) is the ischemic weight, which may vary along thecenterline if ramifications are present. FIG. 18 shows a representationof a vessel segment with multiple bifurcations, corresponding ischemicweights, and longitudinally varying cross-sectional radiuses in ahealthy anatomical model.

Non-healthy segments, such as shown in FIG. 17, have higher ischemiccontribution scores. Higher the severity of the lesion result in higherischemic scores. In this example, the segment is non-healthy due tostenosis, but a same or similar approach may be used for other types ofpathologies (e.g. aneurysm). For a stenosis that stretches along asingle root/branch/leaf segment, the ischemic contribution score iscomputed using the formula:s=f ₄(r(x))w _(l) +f ₅(r(x))w _(l) ²,where f₄ and f₅ are mathematical operators applied to the longitudinallyvarying radius and w_(l) is the weight of the segment. The twocomponents in the contribution score may be used separately as featuresfor training the surrogate model, and/or each component may be dividedinto subcomponents that are then used as features. Other functions maybe used.

FIG. 19 shows one embodiment of a vessel branch or segment that includeshealthy portions and a non-healthy portion, a partially diseased vessel.The ischemic scores are computed separately for these different parts.In a case of bifurcation stenosis as represented in FIG. 20, thestenosis stretches along several root/branch/leaf segments. A separateischemic contribution score is computed for each root/branch/leafsegment of the stenosis pertaining to either the parent or the daughterbranches as represented in FIG. 20. Other approaches, such as using acombined score for the bifurcation, may be used.

Other ischemic features may be computed. For example, based on theischemic contribution scores of individual segments, featuresrepresenting cumulative ischemic contribution scores may be computed atany location in a coronary arterial tree. Various features include:cumulative ischemic contribution score computed from all segments lyingbetween the root segment and the current locations, cumulative ischemiccontribution score computed from the healthy segments lying between theroot segment and the current locations, cumulative ischemic contributionscore computed from the pathologic segments lying between the rootsegment and the current locations, cumulative ischemic contributionscore computed from all segments lying between the current location anda leaf segment (e.g., the path from the current location to the leafsegment may be determined by choosing at each ramification the pathalong the main daughter segment, as determined from a combination ofproperties such as reference radius, total length downstream, and totalnumber of generations downstream), cumulative ischemic contributionscore computed from the healthy segments lying between the currentlocation and a leaf branch, and/or cumulative ischemic contributionscore computed from the pathologic segments lying between the currentlocation and a leaf branch. Additional, different, or fewer ischemicfeatures may be computed.

The ischemic contribution scores and/or the other geometric featuresenlisted above may be computed separately for all pathologic segmentslying upstream and downstream from the current location. Then, thefeatures may be ordered based on a chosen criterion (e.g., ischemiccontribution score or some other feature) and used as an ordered list offeatures.

Features for describing the interaction between branches of vessels maybe extracted. For example, the hemodynamic metric itself, estimatedusing a machine learning algorithm at an upstream location in the vesseltree may be used as a feature for the estimation of the hemodynamicmetric at a downstream location and vice-versa. For example, asdisplayed in FIG. 21, the predicted hemodynamic metric at point A may beused as a feature to predict the hemodynamic metric at point B.

Other features may be defined that account for interaction of flowacross different, possibly not neighboring, vessel segments. Forexample, in FIG. 22, the hemodynamics at points A and B are influencedby the stenosis on the side branch. The presence of the stenosis leadsto a decreased flow in the parent, and hence to a lower pressure in theparent branch. This in turn influences the absolute pressure in thedaughter branch to which point B belongs. Similarly, the presence of thestenosis in the main branch influences the hemodynamics at point C. Thestenosis leads to a lower flow and a lower pressure drop in the parentbranch and, thus to different absolute pressure levels in the sidebranch.

Any approach may be used to account for the interaction between thevessels. A new feature may capture the interaction. Alternatively oradditionally, existing features are modified to account for theiteration. For a new feature, a combination of the features describedfor the different side branches or segments are used for a location on amain branch. Similarly, for a location on the side branch, additionalfeatures computed for the main branches may be used. For example, whengenerating the feature vector for location A in FIG. 23, the totalcontribution score of the upstream side branch with the most severestenosis and the total contribution score of the downstream branch withthe most severe stenosis may be added as features. Any other feature orcombination of features may be used for this purpose.

For modification of other features to account for interaction, theischemic weights of the individual segments are modified in one example.This modification may in turn lead to an adaptation of all featuresbased on ischemic contribution score. The first step is to determine alocal decrease of the ischemic weight separately for each segment:Δw _(l) =f ₆(w _(l) ,s _(l)(w _(l))),where w_(l) is the ischemic weight of the current segment and s_(l) isthe ischemic weight of the current segment, and f₆ is a mathematicaloperator.

Since each segment has a different Δw_(l) value, these changes are usedat a global level to adapt the ischemic weights so as to make sure thatthe original assumptions hold (i.e., the sum of the ischemic weights oftwo daughter branches is equal to the ischemic weight of the parentbranch).

The ischemic weights are globally adapted in a top-down or a bottom-upapproach. For the top-down approach, the weights are adapted from theroot of the tree. Thus, the new ischemic weight of the parent (root)branch is determined as:w _(l) ′=f ₇(w _(l) ,Δw _(l))where w_(l)′ is the new ischemic weight of the parent branch. Next, thenew ischemic weights of the leaf segments downstream from the currentsegment are computed as:

$w_{k}^{\prime} = {\frac{\left( r_{ref} \right)_{k}^{n}}{\sum\limits_{k}\left( r_{ref} \right)_{k}^{n}}{w_{l}^{\prime}.}}$The ischemic weights of the branch lying between the current branch land the leaf branches k are computed as a sum of the ischemia weights ofall downstream leaf segments. Afterwards, the computations are repeatedfor all daughter branches of the current branch, and the process isrepeated recursively until the entire tree is traversed and the leafbranches are reached.

For the bottom-up approach, the ischemic weights of the leaf branchesare adapted as:w _(k) ′=f ₈(W _(k) ,Δw _(k)).Next, the ischemic weight of the parent branch is adapted using:w _(l) ′=f ₉(w _(l) ,Δw _(l) ′,w _(l) ′, . . . ,w _(j)′),where l refers here to the parent branch, while w_(l)′ . . . w_(j)′refer to the new ischemic weights of the immediate daughter branches.This process is repeated recursively until the root branch is reached.

Any of the geometric features may be extracted directly from the medicalimages for application to patient-specific scan data. For example,radius information on a coronary tree is extracted directly from atwo-dimensional projection, without having to reconstruct athree-dimensional vessel. FIG. 24 shows an example x-ray or angiographprojection image from which radii at various locations are extracted.The medical images may be processed, such as filtered, segmented, and/ormasked, or not.

The medical image is a synthetic or artificial image generated fromsynthetic data. For example, the image is a rendering as a projectionfrom a synthetic vessel geometry created from a model. For training, thesynthetic image is used to extract features used for training. In otherembodiments, the image is from a patient, such as by performing amedical scan of the patient. For application of the learned classifier,features are extracted from the image.

The feature extraction is performed on a medical imaging scanner or onanother device, such as an imaging workstation. A processor performs theextraction with or without user input through a user interface having adisplay and user input (e.g., keyboard, mouse, trackball, touch pad,and/or touch screen).

The process of feature extraction from images is fully automated,semi-automated, manual, or a combination of thereof. Under a manualapproach, anatomical or other features are input, annotated or measuredby a human operator or user. For example, the user compiles a list offeatures required for a given hemodynamic metric (e.g., FFR)computation. The list is presented to the user on a display or the userobtains the list from another source. For example, an imaging scanner orworkstation displays a dialog that the user can edit to insert thefeatures. The user may alter the features on the list, such as adding,removing, or changing features. The user then assigns values to thefeatures of the list. The image is used to determine the values for thefeatures. The resulting list of values for the features is stored aspart of the training database 28 or is used for application of themachine-learnt classifier.

In other embodiments, the user compiles a plurality of feature lists,each referring to different parts of the medical image. Each list may beassociated to a different view of the anatomical structure of interestand/or a different spatial region. The user selects one or more parts ofthe image. For each selected part, the system provides a list offeatures. The user may edit the list and assign values to the features.The system combines the lists in a global feature list. The resultingcombined list is stored or used for application. In alternativeembodiments, the lists are maintained separately.

To assist the user, the system automatically proposes one or more viewsof the anatomical structure of interest, cuts (e.g., segment or mask)parts of the medical image, and/or provides measurement tools that allowmeasuring geometrical features of the anatomical structure of interest.FIG. 25 shows an example annotation tool for measuring distance, such asa vessel length (left image) or vessel diameter (right image). A rulerallows computing Euclidean distance between points selected on theimage. FIG. 26 shows an example annotation tool for tracing a centerlineof a vessel or other structure. For example, the user clicks a number ofpoints on the medical image, and the system draws a line connectingthem. The system computes the length of a vessel along the curvilinearabscissa of the centerline.

Other annotation tools may be provided, such as the system providing alist of templates among which the user selects the ones that bestrepresent the anatomical object(s) of interest (e.g. tapering vessels,bifurcations, trifurcations, and/or stenoses with different shapes).FIG. 27 shows an example tool for synthetic representation of theanatomy of interest for both creating the geometry as well as extractingfeatures. The user chooses geometry templates and connects the templatesto represent the whole anatomy. Each geometry template is labeled andcolor coded based on any feature (geometrical, hemodynamics, anatomical,and/or categorical). The geometry templates may be edited by the users(e.g. changing vessel radius, vessel length, vessel curvature, colorcode, or other characteristic). The list of features is automaticallypopulated based on the selected geometry templates. The system mayprovide a same or different list of geometry templates for each part ofthe medical image and/or each view of the object of interest.

Under a semi-automated approach, some of the features may be extractedautomatically by an algorithm, while some others may be annotated oredited, input, and/or corrected by the user. The system provides full orpartial identification of geometry features of the arterial tree or of asubtree. The detected features may be shown on top of the medical imageavailable for further user interaction or annotation. In one embodiment,anatomy is automatically detected by a processor. The user may editand/or correct the detection results. The processor automaticallycomputes the centerline and cross-sectional contours. The user may editand/or correct the detection results. A list of features is displayed tothe user. The user, interacting with the processor, inputs values orindicates the locations of measurements for processor determined valuesto be calculated. After any editing and/or correction by the user, thelist or lists of features with corresponding values are stored or usedin application.

Other embodiments with semi-automatic feature value determination may beused. One or more of various options or differences are provided. Thesystem performs jointly the automatic detection (e.g., myocardium,coronary ostia, and/or main branches) and computation of centerline andcross-sectional contours. The user edits the centerline and thecross-sectional vessel contours by interactively changing their positionand/or shape on the medical image. The user creates new centerlinebranches and additional contours besides the ones automaticallygenerated by the system. The system populates the list of geometricfeatures using both the ones automatically detected and the onesmanually added by the user. The system keeps track of the featurescurrently added to the list and prompts the user to add missingfeatures, if any. The system has a pre-defined ranking of features,based on their effect on the final computed value, and the list offeatures is shown color-coded based on this ranking. One possibleapplication of this is user guidance during feature identification sothat the user may make sure that the most relevant features arecarefully captured. The system computes the hemodynamic metric ofinterest continuously as features are being added to the list, andinteractively shows the resulting value or the metric's sensitivity tothe current feature being added.

In one embodiment, the system displays suggested ranges for eachfeature, based for instance on databases, population averages,literature search, previous data from same patient, or other source. Thesystem compares the current list of features with reference values fromany source and prompts the user to correct and/or confirm features ifthe computed value is outside expected or suggested ranges ofvariations. The system automatically proposes a selection of geometrytemplates representing the anatomical object of interest. The systemdisplays suggested ranges for the parameters of the geometry templates,based for instance on databases, population average, literature search,previous data from same patient, or other source. The systemautomatically fills the list of features, and prompts the user to edit,add, and/or correct the list. When the user adds or edits a feature, allor part of the other features are updated accordingly. Additional,different, or fewer variations for semi-automatic extraction of valuesof features may be provided.

Furthermore, the feature values may be used to indicate for the user onwhich part of the geometry to focus when providing manual input for theextraction of features. For example, if the ischemic contribution scoreof a certain branch is high, then the user should focus on that specificbranch when providing input information (e.g. when segmentation isperformed). FIG. 29 shows an example of a branch with high ischemiccontribution on which the user should focus while preparing the datarequired for feature extraction.

Under a fully-automated approach, an underlying image-processingalgorithm first detects the anatomical region of interest. For example,the algorithm automatically detects the stenosis, coronary vessels,coronary ostium, cardiac chambers, myocardium, trabeculae and papillarymuscles, and/or aorta. Next, the algorithm extracts anatomical featuresfrom the medical image in the detected regions. The system providesfully automatic detection and quantification of the features for thecomputation of the hemodynamic index of interest. The result of theautomatic approach is a complete list of features with populated values.The collection of geometry or other features thus identified may or maynot be enough to reconstruct an accurate three-dimensional geometricalmodel.

Referring again to FIG. 2, values of the hemodynamic metric or metricsof interest are determined in act 16. A value for the flowcharacteristic is determined and stored for each of the examples of thevessel arrangements in the synthetic data. The value of the flow is theground truth used for training the classifier. The values are storedwith the feature for each example in the database 28.

The machine learning maps the input features to a value or values of oneor more hemodynamic metrics. Any hemodynamic metric may be used. Themetric is for a part of the vessel structure or for the overall vesselstructure of interest. Various example metrics include pressure (e.g.,average, instantaneous, time-varying, wave-free interval, averaged overa certain sub-interval of a cardiac cycle, or other), flow rate (e.g.,average, instantaneous, time-varying, wave-free interval, averaged overa certain sub-interval of a cardiac cycle, or other), wall shear stress(e.g., average, instantaneous, or other), oscillatory shear index,vessel wall strain, vessel wall stress, or any combination of the abovedefined by any mathematical operator (e.g., addition, subtraction,multiplication, division, integral, derivative, or other). Examplehemodynamic metrics specifically for the coronary computations includefractional flow reserve (FFR), instantaneous wave free ratio (iFR),ratio of average distal pressure to average proximal pressure (basalPd/Pa), basal stenosis resistance (BSR), hyperemic stenosis resistance(HSR), calcium score, risk of plaque rupture (e.g., separately for eachtype of tissue: fibrous tissue, lipid tissue, necrotic tissue, andcalcified tissue), endothelial dysfunction, or any combination ofthereof.

The hemodynamic metric value or values are extracted for each of thesynthetic examples used in the training data and used for extractingfeatures. The geometric and other features are determined for each setupfor example, and, together with the hemodynamic metric values, thefeatures and values are used to populate the training database. Based onthe representation of the synthetic models (e.g., in vitro or insilico), different methods may be used for extracting the hemodynamicmetric required during the training phase. Flow simulation and/orexperiments are used for the in vitro model. Flow computation, such asbased on computational fluid dynamics, is used for the in silico models.

For the in vitro models, the hemodynamic metric is determined based onmeasurements during a simulation. The pressure, flow, velocity, or otherhemodynamic information used to calculate the value of the hemodynamicmetric are measured. FIG. 28 shows an example in vitro model 23. Themodel 23 includes tubes or other material simulating a vessel. Theshapes of the tubes or by shaping the tubes, the various geometries ofthe vessel may be established. The simplified model 23 of FIG. 28includes an in vitro vessel tree is modeled with tubes, a pumpcirculates a fluid, with properties similar to the ones of human blood,through the in vitro model 23, hydraulic resistances (i.e., flowrestrictors) couple to the terminal in vitro segments to generaterealistic levels of pressure inside the in vitro model, a reservoir forcollecting the fluid, one or several occluders for generatingconstrictions in the in vitro model, and one or more measurement devices(e.g., pressure transducers, flow meters, Doppler probe for measuringvelocity, and/or other sensors) used to determine the hemodynamicmetric. Additional, different, or fewer devices may be provided, such asjoints or clamps for altering branch locations and/or the number ofsegments.

The in vitro model 23 and the flow conditions may be modified innumerous ways to generate a large number of setups. For example, thenumber, position and shape of the occluders is altered. As anotherexample, the resistance at one or more locations is altered. In yetanother example, the operation of the pump is altered. The number ofside branches and any occlusions may be altered. Other alterations ofcombinations of different alterations are used to create differentmodels with corresponding features and resulting flow characteristics.These alterations are used to populate the database with syntheticexamples including the extracted features and hemodynamic metric valueor values for each of many models 23.

For in silico models, there is no experimental table-top set up tomeasure flow. Instead, computational flow dynamics (CFD) or other flowmodeling is used. Any computational approach for modeling the flow ofblood in the human cardiovascular system may be used. Models withdifferent complexities and scales have been proposed, ranging fromlumped (or zero-dimensional-models), one-dimensional models,two-dimensional models, and three-dimensional models with rigid orcompliant walls (e.g., fluid-structure interaction models). Thenonlinear partial-differential equations of these models are solved withfinite difference methods, finite element methods, finite volumemethods, spectral element methods, boundary element method,Lattice-Boltzmann method, other methods, or combinations thereof. Forspecifying the boundary conditions required for performing blood flowcomputations in the synthetic geometries, personalized boundaryconditions (e.g. using allometric scaling laws based on vesselmorphology) or generic boundary conditions may be used. Steady-stateand/or transient flow computations may be used. When personalizing thecomputations based on allometric scaling laws, the personalization mayrefer to any flow state, such as rest, hyperemia or exercise.

Compared to an in vitro setup, for a single synthetic case for in silicomodeling, each location of that case may be used for generating afeature vector in the training database. Moreover, for each syntheticcase, different flow conditions may be imposed and separate featurevectors may be extracted for each flow condition.

Referring again to FIG. 2, machine learning trains the classifier in act14. The input feature vectors and corresponding values of the flowcharacteristics for many vessel arrangements are used in machinelearning. Tens, hundreds, or thousands of examples are generatedsynthetically. The corresponding feature values and hemodynamic metricvalues are used to map the feature values to the metric values. Once thefeatures and the hemodynamic metric of the synthetic vessel trees havebeen extracted, the next step is to train a machine learning algorithmfor predicting the hemodynamic metric.

Any type of machine learning algorithm may be used. The machine learningis supervised, semi-supervised, or unsupervised. Some examples usingsupervised learning include regression, instance-based methods,regularization methods, decision tree learning, Bayesian, kernelmethods, clustering methods, association rule learning, artificialneural networks, dimensionality reduction, and ensemble methods.Probabilistic boosting tree, hierarchal, or other processes may be used.

The machine learning may use all of the input features. Alternatively,the machine learning determines discriminative features and selects afeature set to be used for classifying. A subset of the extractedfeatures may be used for learning, as determined from feature selectionand ranking, feature combination, or other process.

More than one classifier may be created. Since different types ofbranches and regions are present in a vessel tree, different classifiersmay be machine trained for the different branches and/or regions. Forexample, different classifiers are trained for main and side branches,bifurcation regions and single branch regions, different types ofpathologic regions such as different types of single branch stenoticregions (e.g., focal, long, diffuse, restenosis, or other), differenttypes of bifurcation stenoses (e.g. a separate model for eachbifurcation stenosis type in the medina classification), different typesof aneurysms, different types of plaque, different types of total and/orsub-total occlusions, stenotic and regurgitant valves, variouspathologies of the heart (e.g., past infarct or myopathies), ordifferent types of branches (e.g. in case of coronary arterial trees:LM, LAD, LCx, RCA, Diagonal, OM, or other). Since the training is basedon synthetic geometries, a large enough number of training instances maybe generated for each of these different classifiers. Anotherpossibility is to divide the geometry into separate segments (e.g. forcoronary geometries: proximal LAD, mid LAD, and distal LAD) and toextract the features discussed in the previous sections separately foreach segment. Afterwards these features may either be combined intocumulative features or used separately for a single or multiple machinelearning algorithms for predicting a hemodynamic metric of interest.

Once trained, the machine-learnt classifier is instantiated as a matrixor matrices. The matrix maps the values of the input features to valuesof the hemodynamic metric. This mapping is used to predict thehemodynamic metric in 22. In this prediction phase, features areextracted from patient-specific data in act 20. These patient-specificfeatures are input to the machine-learnt classifier, which outputs avalue or values for the hemodynamic metric. For example, based on thefeatures extracted from a medical image of a scan of a patient, thetrained model is applied to compute FFR for that patient.

The machine-learnt classifier may be used in a feedback approach. Whileperforming preparatory steps to extract additional features and/orfeatures for other parts of the vessel, intermediate results may alreadybe computed using the machine learning algorithm and displayed to theuser. This may potentially give useful feedback for obtaining the finalresults. FIGS. 30A and 30B show an example. FIG. 30A shows an example ofa partially processed geometry for which the hemodynamic metric may bepredicted and displayed. FIG. 30B shows the workflow used in this case,which contains a loop in order to continuously generate new predictionswhile the input data is being processed in act 52. This approach isfeasible due to the fact that the prediction of the hemodynamic metricfrom a set of features is almost instantaneous.

Rather than training one classifier, the classifier may be learned as anetwork of different models, where each model works on some subset orthe entirety of the feature space. The outputs from each model may beused as inputs to other models, thereby creating new features. As oneexample, the output of upstream nodes may be used as a feature topredict required quantities at downstream locations, and this proceduremay be applied iteratively to reconstruct the quantity on the entirearterial tree. The output from one model may be used as an input to thesame model to produce recursive model estimates. The classifier may betrained to learn from categorical, discrete, and/or continuous features.The predictive classifier may be a combination of multiple interactingmachine-learnt classifiers, each of which use the same or a differentsubset of features.

Once trained, the machine-learnt classifier or classifiers are used topredict. To predict the flow for a specific patient, medical scan datarepresenting the patient is acquired. The scan data is acquired by amedical scanner and represents the vessel structure of the patient. Forexample, the medical scan data is angiogram data. Computed tomography,magnetic resonance, ultrasound, PET, SPECT, x-ray, combinations thereof,or other type of medical scan data may be acquired. In alternativeembodiments, the scan data is acquired by upload from a memory orreceipt from a transmission. The scan data is specific to a givenpatient, so is from a scan of that patient rather than synthetic data.

The medical scan data represents a three-dimensional region of thepatient. A set of scan data representing intensity at different voxelsdistributed over three dimensions is provided. In other embodiments, themedical scan data is a zero, one, or two-dimensional representation ofthe vessel structured. Two or three-dimensional scan data is processedto create a zero, one, or two-dimensional representation of the vesselstructure of the specific patient.

For prediction, features are extracted from the medical scan data.Features may be extracted from other data for the patient as well.Similarly, replacement features may be provided for features that arenot available for a given patient, such as using an average value.

The approaches discussed above are used to extract the values from themedical scan data and other data for the specific patient. The entireset of features from patient data during prediction is extracted andthen the machine learning algorithm is used to predict a hemodynamicmetric. The preparation of the data for extracting the features usesmanual, semi-automatic, or automatic approaches. For a patient dataset,where some hemodynamic parameters are to be computed, the relevantfeatures are extracted from the patient images and then applies asinputs to the learnt machine learning model.

The features selected by the user are from either the same view, or fromdifferent views of the anatomy. The features may also be selected frommultiple imaging modalities. As an example, if the patient has apre-operative CT scan, some features are selected on the CT scan andsome on the intraoperative angiographic acquisition. These additionalimages may be from any modality, including but not limited to MRI, CT,X-ray angiography, intravenous ultrasound (IVUS) and optical coherencetomography (OCT). The features may contain information about pasthistory of the patient. For example, some of the features are related tostents already in the patient from past percutaneous coronaryintervention (PCI) procedures. If the patient suffers from a severelyenlarged heart, has myocardial scarring from a past infarction, or othercondition, this information may be used as a feature. The predictiveclassifier is adapted to take account of this feature and increaseaccuracy. The features are extracted directly from the medical image orfrom a processed representation of the medical scan data. The processedversion may be a mesh, a mask or probabilistic descriptors of thepresence of different anatomical features.

In act 22 of FIG. 2, the extracted feature values are input to themachine-trained classifier. A processor inputs the values as part ofapplication of the classifier. The machine-trained classifier is trainedonly from synthetic data or from a combination of data from a collectionof patients and synthetic data. For synthetic data, the machine-trainedclassifier is trained from examples of vessel arrangements generatedwith computer modeling, physical modeling, or both computer and physicalmodeling using the in vitro or in silico models and corresponding groundtruth hemodynamic measurements or computations. The features extractedfrom the medical scan data of the patient for application in act 20 areinput to the classifier.

As a result of the input, the processor outputs the value or values forthe hemodynamic metric. The processor applies the machine-trainedclassifier to determine the flow. The flow is output as a value, graph,annotation, display, or image.

The predicted quantity is any hemodynamic quantity, including but notlimited to pressure, velocities and quantities derived from therefrom.For example, the surrogate model predictions are FFR, iFR, CFR, BSR,HSR, basal Pd/Pa, pressure gradients or another quantity. Flow rates,shear stress, time integrals of these quantities, likelihood of plaquerupture, classification of the nature of plaque, or other metric may bepredicted. More than one metric may be predicted.

The predictions are continuous variables, such as the pressure orrelated variables, or categorical variables, such as a discreteprediction of the presence or absence of disease or a discrete gradingof the severity of disease. The predicted indices may be eithercycle-averaged quantities or transient quantities, showing the systolicand diastolic variation. Further, model predictions may be used to inferorgan perfusion and to predict parts of the organ that may bevulnerable. The model predictions may also be used in combination withother imaging data, such as perfusion and stress echo, to improve theimage as well as to identify further features.

In one embodiment, the one or more predicted values are output on adisplay with an image of the vessel structure generated from the medicalscan data. Computed hemodynamic indices may be displayed interactivelyto allow changes in the feature set. If the user chooses to alter thevalue of any feature, the resulting value is reflected in the value ofthe computed indices at all points. The model predictions may also beshown as a ranking of the most severe pathologies, where interventioncould have the most beneficial impact for the patient. For thecoronaries, the model may order the lesions in decreasing order ofseverity. Once one of the lesions is stented, the model may immediatelyupdate the hemodynamic indices, such as FFR, and reorder the remaininglesions according to the new predictions.

The predictions from the model may be used to guide the placement ofinterventional devices such as catheters, pressure wires and for stentdeployment. The predictions may be used to ascertain that the stent hasbeen placed in a manner providing optimal benefit to the patient. Theinteractive nature provided by the efficient prediction from featuresmakes it possible to immediately update the predictions as soon as astent is placed to confirm if the deployment is successful.

In one example output, computed FFR results are visualized on a displayof the medical scanner or on another device, such as an imagingworkstation. A medical image, such as an angiogram, is displayed. Anypoint on the image may be queried (e.g., point and click) for theassociated metric, and the corresponding metric value is shown overlaidto the image. FIG. 31 shows an example where the user selects a point ona root of the vessel structure. As an example, points of interest in thecoronary tree are selected, and the corresponding FFR value is shown inthe image as demonstrated in FIG. 31. The user may activate a “no click”mode, in which case the value of interest is displayed in correspondenceof the cursor by just positioning the cursor on the position ofinterest.

By displaying the value of the metric, other interactions with the usermay be provided. For example, the system provides a touch screenenabling interactions with the anatomical object of interest, such asgestures to rotate, zoom, and pan. Point and touch causes the system todisplay the value of interest at the point of touch. As another example,the system provides an eye-tracking device, so that the value ofinterest is displayed at the location that is being observed by theuser.

Rather than displaying a two-dimensional image or a rendering fromthree-dimensional medical scan data, the arterial tree is represented onthe display as an abstract graph or tree. The graph may be color codedbased on the features of interest. The system may automaticallysynchronize the traversal of the schematic with the traversal of theimage for point-to-point correspondence.

Other outputs of the hemodynamic metric value for a sub-part or lessthan all of the vessel tree may be used. FIG. 32 shows one embodimentwhere a synthetic representation of the anatomy of interest is colorcoded based on the hemodynamic index of interest. The systemsynchronizes the traversal of the image with the traversal of thediagram. By selecting an extracted, coded representation or by selectingthe coded part of the image, the corresponding metric value or valuesare output.

FIG. 33 shows another example output. Based on the extracted features orgeometric structure, the arterial tree is represented as a threedimensional structure that can be visualized and interactively navigatedin a fly-through mode. A similar synthetic three dimensional structuremay also be color coded based on the features of interest. The vesselsurface may be color-coded based on any quantity of interest.

FIG. 34 shows yet another example output. Each vessel is mapped to aplane and represented “unfolded.” In this view, the coronary tree lookslike a two-dimensional tree. Each vessel may be color coded by thefeature or metric value of interest. In this representation, additionalinformation on the vessel is also visualized (e.g. endothelial function,wall shear stress, or plaque burden).

In another embodiment, the coronary tree is mapped to an atlas or apictorial representation of the anatomical structure. The image is coloror otherwise coded based on the value of the feature or metric ofinterest. The system provides an automatic synchronization of thenavigation of the atlas and the image.

FIG. 35 shows another embodiment of an output. Any feature or metric ofinterest is represented by showing one or more particles (glyphs) moving(or fixed) along the centerline (or more generally inside the image).The points are color or otherwise coded based on feature or metric valueof interest. The same particles (glyphs) may be associated with thestatistics of the features or metrics of interest, evaluated at thelocation of the particle. By selecting the particle, the statistics orvalues are shown.

FIG. 36 shows another embodiment of the output. A path (represented as aline) in the vascular tree is shown and color-coded based on the valueof the feature or metric of interest. Either same or different paths maybe determined for different features.

FIG. 37 shows another output. The vessel is represented as athree-dimensional rendering with different cross-section markers. Thecross-section markers are color or otherwise coded based on the value ofthe feature or metric.

In other embodiments, flow pathlines or streamlines are added and colorcoded based on the value of interest. The image of the coronary tree maybe color coded based on any feature extracted during the pre-processingphase, based on any computed feature, or based on the predicted metricvalue. As an example, the computed FFR value is used to color thecoronary tree.

FIG. 38 shows another embodiment of the overall process of FIG. 2. Acts60, 62, 64, and 66 are added for dealing with uncertainty. In act 60,uncertainty is assigned to one or more features. The uncertainty is adistribution of possible values for the feature. For example, the radiusmay be measured as 0.25 cm, but the accuracy or tolerance in themeasurement provides that the radius is between 0.20 cm and 0.30 cm withgreater probability for the values closer to 0.25 cm. Any distributionof possible or probable values may be used, such as a normaldistribution, a distribution from a study, or from another source.

The distributions for a set of one or more uncertain input variables isused in forming the synthetic data. In one example, the confidenceintervals are obtained during the training phase by stochasticallyperturbing the synthetic geometry to obtain a range of predictions.Synthetic examples for each of the possible values are created. As aresult, the machine-learned classifier may output the resulting rangesor distribution of metric values given the uncertainty in the featurevalue. The uncertainty is propagated through a forward model, and theuncertainty for the hemodynamic metric is determined. Alternatively, theuncertainty of the metric value is learned through a machine learningalgorithm based on the extracted features with the distributionreflecting uncertainty of the feature value used as an input.

For prediction, the same features are extracted for a patient-specificgeometry and uncertainty in the input data is specified eitherautomatically or by the user. The user may input or select thedistribution. Using the machine-learnt algorithm, the confidence of theestimated hemodynamic metric is provided. A confidence or probability isprovided for one value of the metric. Alternatively, the predictionsfrom the learnt model may also be ranges or confidence intervals withinwhich the predicted quantity is expected. The predicted confidenceinterval for the patient could be either directly predicted from themodel or estimated from a set of similar anatomies from a saved databaseof synthetic models.

A graph representing the distribution of values of the metric given theuncertainty is output. Any expression of the confidence interval as adistribution of different values of the metric resulting from thedistribution of the input values for a given feature may be used. Theoutput hemodynamic metric includes a confidence or confidence intervalof different values of the metric resulting from the uncertainty invalues of one or more of the input features.

In another embodiment, automatic adaptation is provided. Online machinelearning is used where feedback about accuracy of one or morepredictions are used to add non-synthetic examples to the database 28 sothat repetition of the machine learning may result in a more accurateclassifier. The system is capable of including the effects of knownmeasurements. If the measurement of a hemodynamic parameter for a givenpatient is provided at any location, the system uses this information toimprove the accuracy of any subsequent predictions. Further, the errorin the original prediction at the location where data is provided may beused to improve the mode's future performance. In alternativeembodiments, the machine-trained classifier is used without feedback orupdate.

For feature extraction, the users' corrective actions taken to improveautomatically identified features may be used to improve the featuredetection in the future. The system learns from the user inputs. Theimprovement for feature extraction and/or adaptive learning for theclassifier may be on a global manner or a site-specific manner. Thisallows the system to account for anatomical trends based on patientdemographics.

Other adaptation of the machine learning may occur. If measurements ofthe hemodynamic metric become available, the system may automatically orsemi-automatically identify outlier cases or cases where the value ofthe metric is with a given standard deviation of the norm. These casesare then used to create a new set of synthetic geometries that mimic thefeatures of the outlier, together with the already available trainingset to improve the model predictions. With the updated database 28, theclassifier is trained again.

In addition to anatomy, if flow measurements are also available (e.g.Doppler), then the measurement values are incorporated in the machinelearning approach as ground truth for a given example. The training datais updated with new features characterizing flow as inputs. In theprediction phase, if the measured values of these ‘flow’ relatedfeatures are available, these flow features are used as inputs in thefeature vector. In the absence of flow features, similar patients orsimilar models to the patient are located in the database from thegeometric features to arrive at data-driven estimates of flow indifferent branches. This flow is used as a substitute feature forprediction.

Although a very large number of synthetic cases may be generated fortraining, the examples will not cover all patient-specific cases. Hence,when using the machine-learnt classifier to predict results forpatient-specific data, bad matches between predicted and measuredhemodynamic metrics might appear while validating the machine-learningclassifier. In this case, the workflow displayed in FIG. 39 is used toenrich the database of synthetic cases so as to improve the predictionfor the patient-specific cases that lead to a bad match. The processdisplayed in Error! Reference source not found. may also be performeddirectly on the workstation since the generation of synthetic cases maybe fully automated. In act 70, the case with the bad match isidentified. A distance of the feature vector from the feature vectors ofthe examples is used to identify a bad match. Alternatively, thepredicted value is compared with a measured value to identify the badmatch. In act 72, the reason for the bad match is found. The reason maybe feature values not present, the feature values that are mostdifferent, and/or the feature values most determinative of the flowvalue. In act 74, new synthetic examples with similar features aregenerated and added to the database 28. The value of the hemodynamicmetric for the added examples are computed or measured. In act 76, themachine learning is performed again with the updated or adapted databaseexamples.

In another embodiment, sequential machine learning is used. A sequenceof machine-learnt classifiers is created. For example, a hemodynamicmetric is predicted from geometrical features. That value and otherfeatures are used to predict the same metric using a differentclassifier. Any hierarchy of classifiers and corresponding machinetraining may be used.

In one example, the first machine-learnt classifier is trained withcompletely synthetic data during the training phase. The resultpredicted by the machine-learnt classifier for a patient-specific inputfeature vector may be improved by using patient characteristics. FIG. 39shows improvement using a sequence. First, the geometry is extractedfrom patient-specific medical scan data in act 26, and features areextracted from the vessel geometry in act 20. A flow metric is predictedin act 22 by the classifier trained on purely synthetic data. In act 78,further patient-specific features are extracted, such as age, gender,BMI, measurements from other imaging modalities, or other information.In act 80, a second machine-learnt classifier uses the result predictedby the first classifier as feature, alongside the patientcharacteristics, in order to improve the final prediction. The databaseused for training the second machine-learnt classifier may usenon-synthetic data, such as data from application of the firstclassifier on actual patients where the patient-specific flow ismeasured and used as a ground truth.

Any features may be used for the subsequent classifier. For example,left or right dominance in case of coronary circulation, type of lesionspecified as described for example in the syntax score (e.g., coronarysegment with lesions, type of lesion, medina grading for bifurcationlesions, bifurcation angle, ostial lesion, tortuosity, length of lesion,calcification, thrombus, diffuse disease, or other measure), patientdemographics (e.g., age, gender, BMI, height, mass, smoker/non-smoker,or other), pathological history (e.g., presence of hypertension,presence of hyperlipidemia, diabetes mellitus, angina type(stable/unstable/silent), previous cardiovascular history (stroke,infarct, PCI, stent, CABG, etc.), non-invasive stress tests (e.g. stressecho), peripheral vascular disease, kidney disease, exercise ECG-stresstest, exercise radioisotope test (nuclear stress test, myocardialscintigraphy)), blood biomarkers (e.g., hematocrit, lipoprotein level,triglyceride, or other), medication used in the past or present (e.g.,aspirin, Beta-blocker, Nitrate, Statins, ACE inhibitors, Calcium-channelblockers, or ARBs), measurements extracted using any imaging modality(e.g., MRI→blood velocities, blood flow rates, movement of arterialwall; Doppler→blood velocities; IVUS→plaque characteristics, lumeninformation, eccentricity of lesions; angiography→contrast agentpropagation; and/or echocardiography→myocardial characteristics likemyocardial strain), invasive measurements from catheterization (e.g.,invasive pressure, flow, and/or resistance measurements at any locationin the cardiovascular system), other measurements, or combinationsthereof. Any feature from the first phase of the sequential approach maybe removed from that phase and used only during the second phase.

The sequential machine learning approach may also be used to predict thefuture evolution of the patient. For example, the geometric featurestogether with the predicted hemodynamic metrics and any other featurelisted above may be used for predicting the risk of restenosis. Thesecond classifier is in this case trained on patient evolution dataacquired in the past.

One possibility is to build a database with the patient-specific data ofprevious cases and to use this database during the training of thesequential or second classifier. As described before, during the firststep, the classifier learned on synthetic data is used to generate afirst prediction of the hemodynamic metric. During the second stage, thefeatures extracted for the patient-specific data are used to findsimilar cases in the patient database and a second machine learningalgorithm is applied for predicting the final value of the hemodynamicmetric.

In yet another embodiment, the machine-trained classifier is trained fortherapy planning. Any of various therapies for the vessel may beperformed, such as stenting, cauterizing, cutting, resection, grafting,drug exposure, or other procedure. The therapy is performed to have ahemodynamic effect. The classifier may be used to predict thehemodynamic metric by type, location, and/or amount of therapy.

Similarly, the classifier may be used to determine which of variousabnormalities to treat. The classifier is used to assess the hemodynamiceffect of individual lesions in a vascular tree. FIG. 41 shows acoronary tree with three stenoses. The same approaches and workflows maybe applied to other vascular pathologies. To assess the effect of eachstenosis and thus to determine which stenoses may require PCI, variousapproaches may be used. In one approach, the user marks the stenosis tobe treated. The geometry is modified so as to reflect the placement of astent whose size and positioning is chosen by the user. FIG. 41 showsthe resulting change in geometry. In another approach, the stenoses areautomatically detected. FIG. 42 shows detecting of proximal and distalplanes defining the stenosis. The hemodynamic metrics are adapted so asto remove the effect of each stenosis on the hemodynamics. The initialgeometry does not have to be modified, but instead the metric value isaltered.

Although straight-forward from an algorithmic point of view, the firstapproach has the disadvantage of relying on extensive user interaction.The stenosis is identified, a stent size is chosen, and the effect ofstent placement on the geometry is assessed, all by the user. The secondapproach is fully automated and the user only needs to select thestenosis whose effect on the hemodynamic metric needs to be assessed.For the second approach, the classifier used for assessing thehemodynamic metric has to be modified. If a blood flow modeling approachis used, the pressure drop model may be modified so as to reflect theeffect of a stent on the hemodynamics.

Using the machine learning on synthetic data, another approach isprovided. The feature values extracted and/or the geometry extracted aremodified. FIG. 43 shows a method for modifying one or more features orgeometry to account for therapy in order to decide which stenosis totreat. In acts 82 and 20, the extracted features values or the set offeatures are modified. In act 82, the feature values extracted from thesynthetic geometries are modified to account for the therapy. Duringprediction, the extracted features in act 20 are modified to account forthe therapy. One or both modifications are used. The classifier may betrained on many examples. The extracted features from the patientspecific data are modified to emulate the effects of the planned therapyso that a resulting hemodynamic metric value is predicted. In anotherembodiment, the machine training incorporates likely modifications,creating related synthetic examples and corresponding calculated ormeasured metric values for more accurate training accounting forpossible therapies.

For example, one approach modifies the features related to the ischemiccontribution scores of the stenotic segments:s=f ₄₁(r(x))w _(l) +f ₅₁(r(x))w _(l) ²where f₄₁ and f₅₁ are the modified versions of the operators f₄ and f₅.Furthermore, the ischemic weights of the branches containing thestenosis may also be modified, as a result of a different effect on thetotal contribution score of the corresponding branch or as a result of adifferent interaction between the branches. The modified features,corresponding values, and resulting hemodynamic ground truth are used totrain the classifier. The modified features and corresponding valuesfrom patient-specific data are used to predict from the classifier. Inone example, the modification is of features and values corresponding tothe stenosis being modified to features and values corresponding tohealthy vessel, to a stent, or to results from therapy where less flowrestriction results.

This approach may be further extended in the sense that all possiblepost-stenting scenarios may be evaluated and a comprehensive analysismay be displayed to the user. The stenoses are ranked based on theireffect on the hemodynamic metrics. A suggestion is given to the userregarding the stenoses that require treatment.

FIG. 44 shows an embodiment of a method for addressing differentphysiological states. The physiological states may be any of rest,drug-induced hyperemia (e.g., intracoronary or intravenous), hyperemiagenerated by balloon inflation, exercise, post-treatment, or anotherstate. Machine-learning is used to map from one physiological state toanother. Any of the hemodynamic metrics may be predicted for anypatient-specific state by adapting the features extracted from thesynthetic geometries and by changing the flow conditions in the flowsimulations and/or computations performed for the synthetic geometries.FIG. 44 represents a different approach.

A machine trained classifier is used to map the hemodynamic metricobtained for a certain physiological state of the patient to a differentphysiological state of the patient. A sequential machine learning basedstrategy is applied. The extracted features in act 84 are for a givenstate, so that the value of the hemodynamic metric is predicted for thatstate. In sequence, further features with or without some or all of thefeatures used in act 84 are extracted in act 86. The features extractedare for a different physiological state. A second machine-learntclassifier is trained and used in act 88 to map the results from thefirst physiological state to the second physiological state. This secondclassifier algorithm may rely on any features, such as: geometricfeatures specific to the first physiological state, geometric featuresspecific to the second physiological state, and/or a hemodynamic metricpredicted for the first physiological state. The geometric featuresspecific to the second physiological state may be derived by modifyingthe constants and the operators used, such as in the computation of theischemic weights and ischemic contribution scores.

FIG. 45 shows another embodiment for improving reduced-order modelsusing machine learning. Machine learning approaches may be used toimprove reduced-order models. A full-scale (three-dimensional) bloodflow model provides higher fidelity when computing blood flow comparedto a reduced-order model. For example, the effect of vessel curvature isnot captured in a one-dimensional blood flow model. Additionalcoefficients may be added in the reduced-order model to account for theeffect of properties not captured by the reduced-order model.

To determine the values of these coefficients, a machine learning methodmay be used. A large number of full-scale geometries are first generatedin act 10 and full-scale blood flow computations are performed for thesegeometries in act 90. A set of features describing the property that isnot captured by the reduced-order model are extracted from the geometry,and a set of hemodynamic metrics (e.g. in case of curvature effect thetortuosity features described in a previous section may be used) areextracted from the computational results in act 92. Next, thereduced-order computations are performed in act 94, and the coefficientsin the reduced-order model are adapted so as to match the hemodynamicmetrics extracted from the full-scale model in act 96. The machinelearning algorithm is trained in act 14 so as to be able to predict thevalues of the coefficients solely from the geometric features in act 22.

For example, an additional term may be added in the momentumconservation equation of the one-dimensional model so as to capture theeffect of curvature on the viscous energy losses:

${\frac{\partial{q\left( {x,t} \right)}}{\partial t} + {\frac{\partial}{\partial x}\left( {\alpha\frac{q^{2}\left( {x,t} \right)}{A\left( {x,t} \right)}} \right)} + {\frac{A\left( {x,t} \right)}{\rho}\frac{\partial{p\left( {x,t} \right)}}{\partial x}}} = {{K_{R}\frac{q\left( {x,t} \right)}{A\left( {x,t} \right)}} + {c_{curvature}\frac{q\left( {x,t} \right)}{A\left( {x,t} \right)}}}$The coefficient to be estimated in this case would be c_(curvature)while the hemodynamic metric extracted from the full-scale simulationsis the pressure drop. Other coefficients may be used.

Various figures show methods for predicting a value for hemodynamicmetric or performing other operations. The methods are implemented by amedical diagnostic imaging system, a review station, a workstation, acomputer, a picture and archiving and communications system (PACS)station, a server, combinations thereof, or other device for imageprocessing medical diagnostic data. Different devices may be used fortraining from examples in a database than for predicting. In oneembodiment, the computer for training is described below with respect toFIG. 47. In another embodiment, the system of FIG. 47 predicts with amachine-trained classifier. Other systems may be used for either or bothof training and prediction. A network may be used for providing input,distributed processing, outputting results, or other communications. Amedical scanner provides scan data representing a patient. The scan datais image data or processed data.

The methods are implemented in the order shown or described or adifferent order. Additional, different, or fewer acts may be performed.For example, the acts related to prediction are provided without theacts for training. As another example, the acts for training areprovided without the acts for prediction.

The acts for prediction may be performed in real-time, such as during asurgical procedure, during therapy planning, or during diagnosis by amedical professional. Performing during user interaction allows for moreversatile diagnosis and/or planning. The hemodynamic metric value may bepredicted in less than one minute for real-time performance. In otherembodiments, the acts are performed not in real-time, such a servingresults from a remote sever after a delay of minutes, hours, or days.

Since a machine-learnt classifier is used for predicting the hemodynamicmetric value, the prediction may occur more rapidly than withcomputational flow dynamics. To show this difference, FFR may beanalyzed.

FFR is an invasively measured functional parameter used to characterizethe hemodynamic significance of a coronary artery stenosis. FFR isdefined as the ratio of cycle-averaged pressure distal to the stenosisto the cycle-averaged aortic pressure. Over the years, multiple clinicaltrials have shown that FFR-guided stenting, clinically referred to asPercutaneous Coronary Intervention PCI, is superior toangiography-guided PCI, both in terms of long-term clinical outcomes,decrease in unnecessary revascularization, and cost effectiveness.Although strong clinical data now exists showing the superiority ofFFR-based decision making for coronary stenosis treatment, the use ofFFR is still relatively uncommon. The vast majority of coronarydiagnoses are still based on pure anatomical information observed inmedical images. This has partly been attributed to the requirement ofinducing hyperemia, a condition which increases the blood flow beforemeasuring FFR.

Blood-flow computations, performed using computational fluid dynamics,when used in conjunction with patient-specific anatomical modelsextracted from medical images, have been proposed for diagnosis, riskstratification, and surgical planning. CFD-based blood flow modelingapproaches have been recently applied for evaluating coronary arterialhemodynamics, and estimating FFR. Studies have mainly focused on twotypes of medical image data: computer tomography angiography (CTA) andX-ray coronary angiography (XA). In case of CTA, blood flowcharacteristics are computed in the entire coronary arterial geometry(i.e., left and right coronary artery). Two different approaches may beused: full-order (3D) blood flow modeling where processing time variesbetween 2 and 6 hours, when being performed off-site on supercomputersor reduced-order blood flow modeling where processing time requires10-12 minutes, when being performed on-site on a workstation. In thecase of XA, since the coronary geometry may only be partiallyreconstructed, blood flow characteristics are computed for a subset ofarterial segments. Previous studies reported a processing time whichvaried between 5 minutes and 24 hours. Since XA is invasive, the bloodflow computation should ideally be performed during the procedure, inreal-time or near real-time, so as to enable an immediate diagnosis andguide the patient treatment. These approaches yield good results ascompared with invasively measured FFR. Importantly, the CFD-basedestimation of FFR is able to better discriminate between hemodynamicallysignificant and non-significant coronary artery lesions than the pureanatomical evaluation, when using invasively measured FFR as goldstandard.

A machine-learnt classifier is trained on features extracted fromsynthetic coronary geometries and on the hemodynamic metric of interestFFR, which is computed using a blood flow modeling (CFD) approach. In apreliminary implementation of this set-up, the machine learning-basedFFR predictor produces results on patient-specific data which highlycorrelate with CFD based results for the same data (e.g., correlation:0.9973). FIG. 46 shows the correlation between the two approaches.

Moreover, the machine learning-based approach enables a near real-timeevaluation of coronary hemodynamic indices, requiring a total of 3-7seconds for feature extraction and prediction on a regular desktopcomputer (Intel i7 8 cores, 3.4 GhZ, 8 GB RAM). Hence, the proposedapproach is at least two orders of magnitude faster than reduced-orderblood flow modeling approaches and at least 3 orders of magnitude fasterthan full-order blood flow modeling approaches using CFD. Real-timecomputation of FFR is provided on a standard radiology post-processingworkstation without the need to transfer data offsite or wait for a longtime to assess the results.

Given the advent of Coronary CTA in the emergency department, quickturn-around time for accurate diagnosis (e.g., rule-in or rule-outsignificant coronary disease) is key to improving the overall outcomeand reducing the costs. The clinician may perform changes in the inputdata (e.g. severity of stenosis), motivated by the uncertainty in theinput data, and reevaluate the coronary lesions in real-time.Furthermore, treatment planning may also be performed in near real-time:one or more lesions, marked by the user or chosen automatically, may bevirtually treated (e.g. virtual stent placement), and the remaininglesions may be reevaluated.

Instead of using a hemodynamic quantity as the ground-truth, othermetrics may be used as the ground-truth. As a result of the hemodynamiccomputations, a label may be attached to each location along thecenterlines. The labels may be of any resolution, such as two types‘significant’ and ‘non-significant’, referring to the fact that theupstream lesions are hemodynamically significant or not. Multiple labelsmay be used describing whether the lesion has no effect on thecirculation, a mild effect, an intermediate effect, a severe or a verysevere effect, or other effect. Furthermore, the labels may be based ona perfusion analysis that is performed in junction with the hemodynamiccomputations. A perfusion territory may be associated with each branchand labels of the type ‘Perfusion defect’ or ‘No perfusion defect’ maybe used as ground truth during the training phase.

In another embodiment, the ground truth may be given by the change inluminal radiological attenuation. This approach may be used whensynthetic medical images are used during the training phase, but mayalso be applied if contrast agent propagation analyses are performed forthe synthetic geometries. The change in luminal radiological attenuationmay be described by the change per 10 mm or other length of coronaryartery, and then a linear regression coefficient between intraluminalradiologic attenuation and length from ostium may be computed for use asground truth.

The ground truth may be the outcome from virtual percutaneous coronaryintervention (PCI). In one embodiment, the system performs virtual PCIon each created or detected stenosis. The outcome is computed (e.g., interms of FFR, or percentage perfusion to the downstream districtscompared to the healthy case) and each lesion is graded based oncontribution to the perfusion defect. The ground truth is then a measureof healthy perfusion after virtual PCI, for each location along thecenterline.

Multiple optimization criteria (i.e. cost function that penalizes themismatch between the prediction and the ground-truth) may be considered.One or more of the following metrics: PPV, NPV, specificity,sensitivity, diagnostic accuracy, and correlation may be maximized. Anycombinations of these metric may also be used. For example, thespecificity is maximized while keeping sensitivity less than 90%. Thecost functions may be described in a weighted fashion using two cutoffpoints defining a range of acceptable FFR. For example,min∥FFR_(ML)−FFR_(CFD)∥ over all (FFR_(CFD)<x or FFR_(CFD)>y). In aclinical setting, the lower and upper cutoff points for ML-FFR may bedifferent from an 0.8 cutoff value prescribed for invasive FFR.

Additionally, a cost function may be used for which different weightsmay be attached to different intervals of values of the ground-truthquantity. To achieve high classification accuracy, the values closer tothe clinical cut-off point may have a larger weight than the valuesfurther away from the cut-off (e.g., in case of FFR, the interval0.7-0.9 may have a larger weight than value outside of this range).Furthermore, additional constraints in terms of minimum and maximumvalues may be introduced that reflect the maximum variation of thequantities in clinical practice (e.g. FFR values lie between 0 and 1 inclinical practice). Any of these approaches may be applied for thetraining of any machine learning predictor, irrespective of whether thepredictor being trained is the only predictor used in the application orif sequential machine learning predictors are applied.

FIG. 47 shows a system for hemodynamic determination in medical imaging.The system includes a medical imaging system 11, a processor 13, amemory 15, and a display 16. The processor 13 and the memory 15 areshown separate from the medical imaging system 11, such associated withbeing a computer or workstation apart from the medical imaging system11. In other embodiments, the processor 13 and/or memory 15 are part ofthe medical imaging system 11. In alternative embodiments, the system isa workstation, computer, or server for hemodynamic determination inmedical imaging. For example, the medical imaging system 11 is providedfor acquiring data representing a volume, and a separate database,server, workstation, and/or computer is provided for extracting geometryand/or features and applying a classifier to predict one or morehemodynamic metrics. Additional, different, or fewer components may beused.

The system is used for application. In alternative embodiments, thesystem is used for training and/or generation of the examples in thedatabase.

The computing components, devices, or machines of the system, such asthe medical imaging system 11 and/or the processor 13 are configured byhardware, software, and/or design to perform calculations or other acts.The computing components operate independently or in conjunction witheach other to perform any given act, such as the acts of any of themethods described above. The act is performed by one of the computercomponents, another of the computing components, or a combination of thecomputing components. Other components may be used or controlled by thecomputing components to scan or perform other functions.

The medical imaging system 11 is any now known or later developedmodality for scanning a patient. The medical imaging system 11 scans thepatient for a vessel region. For example, a C-arm x-ray system (e.g.,DynaCT from Siemens), CT like system, or CT system is used. Othermodalities include MR, x-ray, angiography, fluoroscopy, PET, SPECT, orultrasound. The medical imaging system 11 is configured to acquire themedical imaging data representing one or more vessels. The data isacquired by scanning the patient using transmission by the scannerand/or by receiving signals from the patient. The type or mode ofscanning may result in receiving data of just the vessel. Alternatively,data of a volume region is received and the vessel information issegmented from information of other anatomy.

The memory 15 is a buffer, cache, RAM, removable media, hard drive,magnetic, optical, database, or other now known or later developedmemory. The memory 15 is a single device or group of two or moredevices. The memory 15 is within the system 11, part of a computer withthe processor 13, or is outside or remote from other components.

The memory 15 is configured to store medical scan data, extractedgeometry of the vessel tree, extracted features from the medical scandata, geometry or other source, examples (e.g., geometry from syntheticdata, extracted features from the geometry, and ground truth hemodynamicmetric value), and/or other information. For example, the memory 15stores ischemic values, such as a weight and contribution.

The memory 15 is additionally or alternatively a non-transitory computerreadable storage medium with processing instructions. The memory 15stores data representing instructions executable by the programmedprocessor 13 for hemodynamic metric estimation in medical imaging. Theinstructions for implementing the processes, methods and/or techniquesdiscussed herein are provided on computer-readable storage media ormemories, such as a cache, buffer, RAM, removable media, hard drive orother computer readable storage media. Computer readable storage mediainclude various types of volatile and nonvolatile storage media. Thefunctions, acts or tasks illustrated in the figures or described hereinare executed in response to one or more sets of instructions stored inor on computer readable storage media. The functions, acts or tasks areindependent of the particular type of instructions set, storage media,processor or processing strategy and may be performed by software,hardware, integrated circuits, firmware, micro code and the like,operating alone or in combination. Likewise, processing strategies mayinclude multiprocessing, multitasking, parallel processing and the like.In one embodiment, the instructions are stored on a removable mediadevice for reading by local or remote systems. In other embodiments, theinstructions are stored in a remote location for transfer through acomputer network or over telephone lines. In yet other embodiments, theinstructions are stored within a given computer, CPU, GPU, or system.

The processor 13 is a general processor, digital signal processor,three-dimensional data processor, graphics processing unit, applicationspecific integrated circuit, field programmable gate array, digitalcircuit, analog circuit, combinations thereof, or other now known orlater developed device for processing data. The processor 13 is a singledevice, a plurality of devices, or a network. For more than one device,parallel or sequential division of processing may be used. Differentdevices making up the processor 13 may perform different functions, suchas extracting geometry or feature values by one device and computationof flow quantities by another device. In one embodiment, the processor13 is a control processor or other processor of the medical imagingsystem 11. The processor 13 operates pursuant to stored instructions toperform various acts described herein.

The processor 13 is configured to extract geometry, extract featurevalues, interact with the user in extraction, apply features to amachine-trained predictor, and generate an image or other output. Inembodiment, the processor 13 is configured to modify one or morefeatures or feature values to emulate a geometry being in atherapeutically corrected state from an abnormal state. By modifying thefeatures, the hemodynamic operation of the vessel after therapy may bepredicted. The processor 13 is configured to apply the features,including any modified features or features with uncertainty, to amachine-trained predictor trained with training data of examples ofvessels. The machine-trained predictor may be trained from trainingexamples in the therapeutically corrected state for prediction oftherapy results. For therapy planning, the application is repeated bythe processor 13 multiple times for different modifications of thefeature or features associated with different therapeutically correctedstates. For uncertainty, the application is performed once where thepredictor is trained on uncertainty information or is performed multipletimes to determine a distribution of the hemodynamic metric values giventhe uncertainty of the input feature value.

The processor 13 is configured to output a prediction. By application ofthe input feature vector to the machine-learnt predictor, the predictoroutputs a prediction or estimate of the hemodynamic variable, such asFFR. The output prediction is in the form of text, graph, color coding,or other representation.

The display 16 is a CRT, LCD, plasma, projector, printer, or otheroutput device for showing an image. The display 16 displays the quantityor quantities output by the processor 13. The quantities may bedisplayed in a chart, graph, and/or on an image. The display 16 isconfigured by display values to indicate the value of the hemodynamicmetric. The value may be displayed in association with the geometry,features, and/or an image. In one embodiment, the value of thehemodynamic metric is displayed with an image representing atherapeutically corrected state. In an additional or alternativeembodiment, the uncertainty associated with the value of the metric isdisplayed, such as displaying the value as an uncertainty interval.

While the invention has been described above by reference to variousembodiments, it should be understood that many changes and modificationscan be made without departing from the scope of the invention. It istherefore intended that the foregoing detailed description be regardedas illustrative rather than limiting, and that it be understood that itis the following claims, including all equivalents, that are intended todefine the spirit and scope of this invention.

We claim:
 1. A method for hemodynamic determination in medical imaging,the method comprising: acquiring medical scan data representing a vesselstructure of a patient; extracting a set of features from the medicalscan data; modifying a first of the features of the set, the modifyingrepresenting a change to the vessel structure due to therapy; assigningan uncertainty to the first feature of the set; inputting, by aprocessor, the features to a machine-trained classifier, the featuresincluding the first feature after the modifying and with theuncertainty, the machine trained classifier trained only from syntheticdata not specific to any patients; and outputting, by the processor withapplication of the machine-trained classifier, a hemodynamic metric witha confidence interval for different values of the hemodynamic metric. 2.The method of claim 1 wherein acquiring comprises acquiring angiographydata.
 3. The method of claim 1 wherein acquiring comprises acquiringwith the medical scan data comprising a two or three-dimensionalrepresentation of the vessel structure.
 4. The method of claim 1 whereinextracting the set of the features comprises: extracting geometricalfeatures of the vessel structure; and extracting the features of one ormore abnormalities of the vessel structure.
 5. The method of claim 1wherein extracting the set of the features comprises extractingfunctional features representing operation of the vessel structure,wherein the machine-trained classifier is trained from virtualrepresentations of the operation of the vessel.
 6. The method of claim 1wherein extracting the set of the features comprises extracting anischemic weight and an ischemic contribution score, the ischemiccontribution score being a function of the ischemic weight.
 7. Themethod of claim 6 wherein extracting the ischemic weight comprisescomputing branch ischemic weights as a function of a global ischemicweight.
 8. The method of claim 6 wherein extracting the ischemiccontribution score comprises computing the ischemic contribution scoreas a function of the ischemic weight and a radius.
 9. The method ofclaim 1 wherein extracting the set of the features comprises extractingbranch interaction features.
 10. The method of claim 1 wherein modifyingthe first feature comprises replacing a feature value corresponding to aflow restriction with a feature value corresponding to mitigation of theflow restriction, and wherein the synthetic data models the change fromthe flow restriction to the stent.
 11. The method of claim 1 whereinassigning the uncertainty comprises estimating a distribution of inputvalues for the first feature, and wherein outputting comprisesoutputting the confidence interval as a distribution of the differentvalues of the hemodynamic metric resulting from the distribution of theinput values for the first feature, the machine-trained classifiertrained with uncertainty information.
 12. The method of claim 1 whereininputting comprises inputting to the machine-trained classifier trainedonly from the synthetic data, the synthetic data comprising an in vitromodel with a ground truth of the hemodynamic metric measured form the invitro model.
 13. The method of claim 1 wherein inputting comprisesinputting to the machine-trained classifier trained only from thesynthetic data, the synthetic data comprising an in silico model with aground truth of the hemodynamic metric computed with computation fluiddynamics.
 14. The method of claim 1 wherein inputting comprisesinputting a sub-set of the set of features, the sub-set for a sub-partof the vessel structure, and wherein outputting comprises outputting thehemodynamic metric for the sub-part of the vessel tree; and furthercomprising subsequently repeating the inputting and outputting forremaining features of the set for another part of the vessel structure.15. The method of claim 1 wherein outputting comprises outputting atleast one of the different values on a display with an image of thevessel structure generated from the medical scan data.
 16. The method ofclaim 1 further comprising predicting another value of the hemodynamicmetric with another machine-trained classifier using at least one of thedifferent values and patient characteristics as input features.
 17. Themethod of claim 1 wherein inputting comprises inputting to themachine-trained classifier trained from the synthetic data, thesynthetic data comprising examples generated by regular variation of anin vitro model, in silico model, or both in vitro and in silico models.18. A method for hemodynamic determination in medical imaging, themethod comprising: generating a plurality of examples of vesselarrangements with computer modeling, physical modeling, or both computerand physical modeling, the examples being synthetic so as to be notspecific to any patients; storing a value for a flow characteristic foreach of the examples of the vessel arrangements; extracting an inputfeature vector from each of the examples; and training, with machinelearning, a mapping from the input feature vectors and the values forthe flow characteristic of the vessel arrangements.
 19. The method ofclaim 18 wherein generating comprises perturbing the computer modeling,physical modeling, or both in a systematic pattern.
 20. A system forhemodynamic determination in medical imaging, the system comprising: ascanner configured to scan a vessel of a patient; a memory configured tostore a plurality of features of the vessel of the patient, the featuresdetermined from the scan of the vessel; a processor configured to modifya first feature of the features from an abnormal state to atherapeutically corrected state, to apply the features including thefirst feature as modified to a machine-trained predictor trained withtraining data of examples of vessels in the therapeutically correctedstate, at least some of the examples of the vessels used to train themachine-trained predictor are synthetic examples not specific to anypatients, and to output a prediction of a value of a hemodynamicvariable based on the application of the features to the machine-trainedpredictor; and a display configured to indicate the value of thehemodynamic variable in association with the therapeutically correctedstate.
 21. The system of claim 20 wherein the first feature comprises anischemic value and wherein the application is repeated multiple timesfor different modifications of the first feature associated withdifferent therapeutically corrected states.